OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 3:08 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 13 posts ] 
Author Message
 Post subject: [Bochs] PXE TFTP Issues
PostPosted: Mon Oct 05, 2015 7:05 pm 
Offline
Member
Member
User avatar

Joined: Fri Jan 16, 2009 8:34 pm
Posts: 284
Location: Louisiana, USA
So after several days of trying to get a working environment for PXE, and testing with known working Ubuntu Installer, I have ran into a road block (with Bochs).

On Bochs I get the following output:
Code:
Booting from Network [iPXE]...
iPXE starting execution...ok
iPXE initialising devices...ok


iPXE v1.0.0-591-g7aee315
iPXE 1.0.0+ -- Open Source Network Boot Firmware -- http://ipxe.org
Features: HTTP iSCSI DNS TFTP AoE bzImage COMBOOT ELF MBOOT PXE PXEXT
net0: b0:c4:20:00:00:00 using E1000_DEV_ID_82540EM on PCI00:03.0 (open)
  [Link:up, TX:0 TXE:0 RX:0 RXE:0]
DHCP (net0 b0:c4:20:00:00:00)...... ok
net0: 192.168.1.165/255.255.255.0 gw 192.168.1.1
Next server: 192.168.1.1
Filename: pxeloader
tftp://192.168.1.1/pxeloader... ok
Getting PXE INFO....DONE!
  Client IP Address 192.168.001.165
  Server IP Address 192.168.001.001
  Relay IP Address 000.000.000.000
Opening TFTP File: pxeboot.cfg...UNKNOWN ERROR 0x0006.

Now on real hardware, this works flawlessly.

Here are parts of the code that I am currently working on:
Code:
mov si, BOOTCFG
call TFTP_OPEN
...
PXE_TFTP_OPEN equ 0x0020
...
;ds:si = filename
TFTP_OPEN:
   mov eax, DWORD [ServerIP]
   mov DWORD [PXENV_TFTP_OPEN_t.SIP], eax
   mov eax, DWORD [RelayIP]
   mov DWORD [PXENV_TFTP_OPEN_t.GIP], eax
   mov di, PXENV_TFTP_OPEN_t.Filename
   .SetFileName:
      lodsb
      test al, al
      jz .FNDone
      mov BYTE [ds:di], al
      inc di
      jmp .SetFileName
   .FNDone:
      mov BYTE [ds:di], 0
   mov ax, 69
   xchg al, ah
   mov WORD [PXENV_TFTP_OPEN_t.Port], ax ; tftp udp port number (BigEndin)
   mov WORD [PXENV_TFTP_OPEN_t.PacketSize], 0x1000 ; 4096 byte packets

   mov di, PXENV_TFTP_OPEN_t
   mov bx, PXE_TFTP_OPEN
   call usePXE
   test ax, ax
   mov ax, WORD [PXENV_TFTP_OPEN_t.Status]
   jnz TFTP_ERROR
   cld
   ret
...
PXENV_TFTP_OPEN_t:
   .Status dw 0
   .SIP dd 0
   .GIP dd 0
   .Filename times 128 db 0
   .Port dw 0
   .PacketSize dw 0
...
usePXE:
   push ds
   push di
   push bx
   call far [PXEAPI]
   add sp, 6
   ret
Here is my Bochs Config (part):
Code:
pci: enabled=1, chipset=i440fx, slot1=pcivga, slot2=ne2k
ne2k: enabled=1, mac=b0:c4:20:00:00:00, ethmod=linux, ethdev=eth0
ne2k: ioaddr=0x300, irq=9, mac=b0:c4:20:00:00:00, bootrom="./pxe-ne2k_pci.rom"
And Also Tried
Code:
pci: enabled=1, chipset=i440fx, slot1=pcivga, slot2=e1000
e1000: enabled=1, mac=b0:c4:20:00:00:00, ethmod=linux, ethdev=eth0
e1000: ioaddr=0x300, irq=9, mac=b0:c4:20:00:00:00, bootrom="./pxe-e1000.rom"
Couln't find a bootrom for pcipnic - so did not test it.

My HW tests have been on 4 completely different computers (no similar HW at all... each with different NICs...)

And my research tells me PXE-E06 = Option ROM requires DDIM support or PXENV_STATUS_OUT_OF_RESOURCES.

Any suggestions on how to get Bochs working with PXE and TFTP? As it will not even boot ubuntu installer after the boot menu.



If it helps Bochs Setup:
Code:
./configure --enable-smp --enable-cpu-level=6 --enable-all-optimizations --enable-x86-64 --enable-pci --enable-vmx --enable-debugger --enable-disasm --enable-debugger-gui --enable-logging --enable-fpu --enable-3dnow --enable-sb16 --enable-cdrom --enable-x86-debugger --enable-iodebug --enable-plugins --disable-docbook --with-sdl --with-x --with-x11 --with-term --enable-pnic --enable-ne2000 --enable-e1000 --enable-clgd54xx --enable-voodoo --enable-monitor-mwait --enable-avx


NOTE: If you read this before I edited the timeout was caused by port number being littleendian not bigendian, but the issue still remains Bochs will not use TFTP after bootfile.

_________________
BOS Source Thanks to GitHub
BOS Expanded Commentary
Both under active development!
Sortie wrote:
  • Don't play the role of an operating systems developer, be one.
  • Be truly afraid of undefined [behavior].
  • Your operating system should be itself, not fight what it is.


Last edited by BASICFreak on Tue Oct 06, 2015 1:27 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: [Emulators] PXE TFTP Issues
PostPosted: Tue Oct 06, 2015 12:29 pm 
Offline
Member
Member
User avatar

Joined: Fri Jan 16, 2009 8:34 pm
Posts: 284
Location: Louisiana, USA
Well, I have tested my code on two more emulators:

QEMU: same spot, error code PXE-E3C Access Violation.

Oracle VM: same exact thing as Bochs - though this is the only one that will fully start Ubuntu installer.

Is there any advice? I'm assuming my code is the issue - but I still cannot understand why it works on so many PCs and no emulators...



And no, there is no Access Violation - I can get the file fine from tftp CLI.:
Code:
[cut]/BOS/0.0.4$ tftp
tftp> connect 192.168.1.1
tftp> get pxeboot.cfg
Received 40 bytes in 0.0 seconds
tftp> quit
[cut]/BOS/0.0.4$ cat pxeboot.cfg
Test Of TFTP File Load Over PXE BIOS!

NOTE: The bootrom(s) I used on Bochs are from QMEU's GitHub


UPDATE: I have compiled both gPXE and iPXE and both have the same result in Bochs of Error 0x0006.

_________________
BOS Source Thanks to GitHub
BOS Expanded Commentary
Both under active development!
Sortie wrote:
  • Don't play the role of an operating systems developer, be one.
  • Be truly afraid of undefined [behavior].
  • Your operating system should be itself, not fight what it is.


Top
 Profile  
 
 Post subject: Re: [Bochs] PXE TFTP Issues
PostPosted: Sat Oct 24, 2015 6:15 pm 
Offline
Member
Member
User avatar

Joined: Fri Jan 16, 2009 8:34 pm
Posts: 284
Location: Louisiana, USA
I'm bumping this thread one last time.

I have finished my PXE loader (no thanks to any of the emulators...)

I have used 3 different BIOSes for each of the virtual cards on both Bochs and Qemu, no matter how I slice it I cannot get past TFTP Open

And because my code is completed I'll post the full source of all PXE functions:
Code:
%ifndef __PXEBASEAPI_INC__INCLUDED__
%define __PXEBASEAPI_INC__INCLUDED__

[bits 16]

PXE_GET_CACHED_INFO equ 0x0071
PXE_TFTP_OPEN equ 0x0020
PXE_TFTP_READ equ 0x0022
PXE_TFTP_CLOSE equ 0x0021

; NOTE THESE ARE NOT REGISTER SAFE!

init_PXE:
   mov cx, 22; 44/2
   mov di, PXEENV
   mov si, bx
   .copyPXEENV:
      mov ax, WORD [es:si]
      mov WORD [ds:di], ax
      add si, 2
      add di, 2
      dec cx
      jnz .copyPXEENV
   cmp WORD [PXEENV.Version], 0x0201
   jge .NewerPXE
   mov ax, WORD [PXEENV.RMEntryOffset]
   mov WORD [PXEAPIOffset], ax
   mov ax, WORD [PXEENV.RMEntrySegment]
   mov WORD [PXEAPISegment], ax
   jmp .Contine
   .NewerPXE:
      push es
      mov ax, WORD [PXEENV.PXEPtrSegment]
      mov es, ax
      mov si, WORD [PXEENV.PXEPtrOffset]
      mov di, PXE
      mov cx, 12; 24/2
      .copyPXE:
         mov ax, WORD [es:si]
         mov WORD [ds:di], ax
         add si, 2
         add di, 2
         dec cx
         jnz .copyPXE
      pop es
      mov ax, WORD [PXE.RMEntryOffset]
      mov WORD [PXEAPIOffset], ax
      mov ax, WORD [PXE.RMEntrySegment]
      mov WORD [PXEAPISegment], ax
   .Contine:
      mov bx, PXE_GET_CACHED_INFO
      mov di, PXENV_GET_CACHED_t
      call usePXE
      test ax, ax
      mov ax, WORD [PXENV_GET_CACHED_t]
      jnz TFTP_ERROR

      mov ax, WORD [PXENV_GET_CACHED_t.BufferSeg]
      mov bx, WORD [PXENV_GET_CACHED_t.BufferOff]
      push es
      mov es, ax
   
      mov eax, DWORD [es:bx + 16]
      mov DWORD [ClientIP], eax
      mov eax, DWORD [es:bx + 20]
      mov DWORD [ServerIP], eax
      mov eax, DWORD [es:bx + 24]
      mov DWORD [RelayIP], eax

      pop es

      ret

;ds:si = filename
TFTP_OPEN:
   push si
   mov eax, DWORD [ServerIP]
   mov DWORD [PXENV_TFTP_OPEN_t.SIP], eax
   mov eax, DWORD [RelayIP]
   mov DWORD [PXENV_TFTP_OPEN_t.GIP], eax
   mov di, PXENV_TFTP_OPEN_t.Filename
   .SetFileName:
      lodsb
      test al, al
      mov BYTE [ds:di], al
      jz .FNDone
      inc di
      jmp .SetFileName
   .FNDone:
      mov ax, 69
      xchg al, ah
      mov WORD [PXENV_TFTP_OPEN_t.Port], ax ; tftp udp port number !!!BIGENDIAN!!!
      mov WORD [PXENV_TFTP_OPEN_t.PacketSize], 0x200 ; 512 byte packets
      mov di, PXENV_TFTP_OPEN_t
      mov bx, PXE_TFTP_OPEN
   call usePXE
   pop si
   test ax, ax
   jnz .ERROR
   ret
   .ERROR:
      mov ax, WORD [PXENV_TFTP_OPEN_t.Status]
      jmp TFTP_ERROR

TFTP_CLOSE:
   mov di, PXENV_TFTP_CLOSE_t
   mov bx, PXE_TFTP_CLOSE
   call usePXE
   test ax, ax
   mov ax, WORD [PXENV_TFTP_CLOSE_t.Status]
   jnz TFTP_ERROR
   ret

;ds:di = buffer
;RET AX = PACKET SIZE RECIVED
TFTP_READ_PACKET:
   mov WORD [PXENV_TFTP_READ_t.BufferOff], di
   mov WORD [PXENV_TFTP_READ_t.BufferSeg], ds
   mov di, PXENV_TFTP_READ_t
   mov bx, PXE_TFTP_READ
   call usePXE
   test ax, ax
   jnz .ERROR
   mov ax, WORD [PXENV_TFTP_READ_t.BufferSize]
   ret
   .ERROR:
      mov ax, WORD [PXENV_TFTP_OPEN_t.Status]
      jmp TFTP_ERROR

; DS:DI = PACKET
; BX = PXE Opcode
usePXE:
   push ds
   push di
   push bx
   call far [PXEAPI]
   add sp, 6
   ret

;input ax = error code
TFTP_ERROR:
   push ERROR.halt ; Push return address.
   cmp ax, 0x01
   je .PXEE01
   cmp ax, 0x32
   je .PXEE32
   cmp ax, 0x35
   je .PXEE35
   cmp ax, 0x36
   je .PXEE36
   cmp ax, 0x38
   je .PXEE38
   cmp ax, 0x39
   je .PXEE39
   cmp ax, 0x3A
   je .PXEE3A
   cmp ax, 0x3B
   je .PXEE3B
   cmp ax, 0x3C
   je .PXEE3C
   cmp ax, 0x3F
   je .PXEE3F
   mov si, TFTP_ERROR_CODES.PXEExx
   call puts
   jmp PrintHex
   .PXEE01:
      mov si, TFTP_ERROR_CODES.PXEE01
      jmp puts
   .PXEE32:
      mov si, TFTP_ERROR_CODES.PXEE32
      jmp puts
   .PXEE35:
      mov si, TFTP_ERROR_CODES.PXEE35
      jmp puts
   .PXEE36:
      mov si, TFTP_ERROR_CODES.PXEE36
      jmp puts
   .PXEE38:
      mov si, TFTP_ERROR_CODES.PXEE38
      jmp puts
   .PXEE39:
      mov si, TFTP_ERROR_CODES.PXEE39
      jmp puts
   .PXEE3A:
      mov si, TFTP_ERROR_CODES.PXEE3A
      jmp puts
   .PXEE3B:
      mov si, TFTP_ERROR_CODES.PXEE3B
      jmp puts
   .PXEE3C:
      mov si, TFTP_ERROR_CODES.PXEE3C
      jmp puts
   .PXEE3F:
      mov si, TFTP_ERROR_CODES.PXEE3F
      jmp puts

PXEENV:
   .Signiture db 0, 0, 0, 0, 0, 0
   .Version dw 0
   .Length db 0
   .CheckSum db 0
   .RMEntry:
   .RMEntryOffset dw 0
   .RMEntrySegment dw 0
   .PMOffset dd 0
   .PMSelector dw 0
   .StackSeg dw 0
   .StackSize dw 0
   .BCCodeSeg dw 0
   .BCCodeSize dw 0
   .BCDataSeg dw 0
   .BCDataSize dw 0
   .UNDIDataSeg dw 0
   .UNDIDataSize dw 0
   .UNDICodeSeg dw 0
   .UNDICodeSize dw 0
   .PXEPtr:
   .PXEPtrOffset dw 0
   .PXEPtrSegment dw 0

PXE:
   .Signiture db 0, 0, 0, 0
   .Length db 0
   .CheckSum db 0
   .Revision db 0
   .Reserved db 0
   .UNDIROMID dd 0
   .BCROMID dd 0
   .RMEntry:
   .RMEntryOffset dw 0
   .RMEntrySegment dw 0
   .PMEntry dd 0


PXENV_GET_CACHED_t:
   .Status dw 0
   .PacketType dw 3
   .BufferSize dw 0x1000
      .BufferOff dw 0x1000
      .BufferSeg dw 0
   .BufferLimit dw 0

PXENV_TFTP_OPEN_t:
   .Status dw 0
   .SIP dd 0
   .GIP dd 0
   .Filename times 128 db 0
   .Port dw 0
   .PacketSize dw 0

PXENV_TFTP_READ_t:
   .Status dw 0
   .PacketNumber dw 0
   .BufferSize dw 0
   .BufferOff dw 0
   .BufferSeg dw 0

PXENV_TFTP_CLOSE_t:
   .Status dw 0

PXEAPI:
PXEAPIOffset dw 0
PXEAPISegment dw 0

ClientIP dd 0
ServerIP dd 0
RelayIP dd 0

TFTP_ERROR_CODES:
   .PXEE01 db "PXE-E01: TFTP error - File not found. or Access violation.", 0
   .PXEE32 db "PXE-E32: TFTP open timeout.", 0
   .PXEE35 db "PXE-E35: TFTP read timeout.", 0
   .PXEE36 db "PXE-E36: Error received from TFTP server.", 0
   .PXEE38 db "PXE-E38: TFTP cannot open connection.", 0
   .PXEE39 db "PXE-E39: TFTP cannot read from connection.", 0
   .PXEE3A db "PXE-E3A: TFTP too many packages.", 0
   .PXEE3B db "PXE-E3B: TFTP error - File not found.", 0
   .PXEE3C db "PXE-E3C: TFTP error - Access violation.", 0
   .PXEE3F db "PXE-E3F: TFTP packet size is invalid.", 0
   .PXEExx db "UNKNOWN ERROR! ", 0

%endif
And the strap I use to actually read the file into memory
Code:
%ifndef __TFTP_INC__INCLUDED__
%define __TFTP_INC__INCLUDED__

[bits 16]
TFTP_ReadFile:
   pusha
   db 0x66
      push edx
   push si
   mov si, LOADMSG
   call puts
   pop si
   call puts
   mov DWORD [FileSize], 0
   call TFTP_OPEN
   .ReadLoop:
      mov si, WAITMSG
      call puts
      mov di, 0x1000
      call TFTP_READ_PACKET
      db 0x66
         xor ecx, ecx
      mov cx, ax
      db 0x66
         pop edx
      db 0x66
         mov edi, edx
      db 0x66
         push edx
      db 0x66
         xor eax, eax
      mov ax, WORD [PXENV_TFTP_READ_t.PacketNumber]
      dec ax
      db 0x66
         xor edx, edx
      mov dx, WORD [PXENV_TFTP_OPEN_t.PacketSize]
      db 0x66
         mul edx
      db 0x66
         add edi, eax
      db 0x66
         push ecx
      db 0x66
         mov esi, 0x1000
      call memcpy
      db 0x66
         pop ecx
      add DWORD [FileSize], ecx
      cmp cx, WORD [PXENV_TFTP_OPEN_t.PacketSize]
      je .ReadLoop
   call TFTP_CLOSE
   db 0x66
      pop edx
   mov si, DONEMSG
   call puts
   popa
   mov eax, DWORD [FileSize]
   ret

LOADMSG db "Loading TFTP File ", 0
FileSize dd 0

%endif


So, all source interfacing the BootROM is provided.
Can anyone explain how to get the emulators to function properly?

I really do not want to fix (rewrite) my disk boot loader (this week, maybe next) - I improved upon a lot of the osloader and changed the Kernel's requirements - plus my (realmode) FAT driver is an error waiting to happen (using 32-bit values to start the math - then only using 16-bit arithmetic) #-o

It is very difficult, though not impossible, to debug on real hardware. I miss watching my register values and everything else :(

But the good news (at least for me) is I now know my scheduler, IPC, and memory management work properly on real hardware. No triple faults, no random lockups, and everything ran as programmed for the whole duration on 5 different systems. :D =D>



NOTE TO ANYONE WHO USES PXE:
Not all hardware will return its "negotiated" packet size on TFTP_OPEN, it will remain what you requested, though may be set to 0x200 (512 bytes) without informing you!

_________________
BOS Source Thanks to GitHub
BOS Expanded Commentary
Both under active development!
Sortie wrote:
  • Don't play the role of an operating systems developer, be one.
  • Be truly afraid of undefined [behavior].
  • Your operating system should be itself, not fight what it is.


Top
 Profile  
 
 Post subject: Re: [Bochs] PXE TFTP Issues
PostPosted: Sat Oct 24, 2015 6:54 pm 
Offline
Member
Member

Joined: Tue Nov 08, 2011 11:35 am
Posts: 453
Why do you manually put OperandSize prefix (db 0x66)?
Correct translator (such as FASM and, possibly, NASM) puts them automatically according to operands and assumed CPU mode that you specify using "bits" directive.

What's this:
Code:
      db 0x66
         xor ecx, ecx
      mov cx, ax
?
Did you want to do this:
Code:
movzx ecx, ax
?

Your code is strange for sure.


Top
 Profile  
 
 Post subject: Re: [Bochs] PXE TFTP Issues
PostPosted: Sat Oct 24, 2015 7:50 pm 
Offline
Member
Member
User avatar

Joined: Fri Jan 16, 2009 8:34 pm
Posts: 284
Location: Louisiana, USA
Nable wrote:
Why do you manually put OperandSize prefix (db 0x66)?
Correct translator (such as FASM and, possibly, NASM) puts them automatically according to operands and assumed CPU mode that you specify using "bits" directive.

What's this:
Code:
      db 0x66
         xor ecx, ecx
      mov cx, ax
?
Did you want to do this:
Code:
movzx ecx, ax
?

Your code is strange for sure.

The reason is simple, nasm assumes I want "xor cx, cx" and will not place the opcode 66h and unless I did "mov ecx, DWORD 0" which without DWORD nasm will assume "mov cx, 0" without the opcode.
I've had to step threw a-lot of code to find that, and it is only in the 16-bit directive - the 32-bit directive does what the manual (I assume) says. (I guess I never told it the CPU, but [bits 16] should be enough... right? In real mode the CPU shouldn't matter much, after 386 that is.)

And I have not considered movzx (I had to look it up just now) - I'll probably change these when I comment my code in the morning. (As I said in a previous post [another thread] my ASM knowledge is improving - it's by no means great)

Either way the math still makes ecx = (0x00000000 | ax)
So, unfortunately this cannot be the issue - with end result at least. Plus it does not get that far PXE replies with error code in TFTP_OPEN which is before this is ever executed. (Error codes are in above posts)

_________________
BOS Source Thanks to GitHub
BOS Expanded Commentary
Both under active development!
Sortie wrote:
  • Don't play the role of an operating systems developer, be one.
  • Be truly afraid of undefined [behavior].
  • Your operating system should be itself, not fight what it is.


Top
 Profile  
 
 Post subject: Re: [Bochs] PXE TFTP Issues
PostPosted: Sat Oct 24, 2015 8:34 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

I'm not sure if this is the problem, but...

While writing my PXE loader I found that the version of iPXE used by VirtualBox is a buggy piece of crap; and I suspect that Bochs and Qemu are using the same buggy version. I don't know if iPXE has been fixed yet.

The problem is that iPXE ignores your "PXENV_TFTP_OPEN_t.PacketSize" value and gives you the network's max. packet size (e.g. about 1500 bytes for ethernet), and if you've got a smaller buffer a subsequent "TFTP read" would overflow your buffer and trash data. Correct behaviour is that "TFTP open" chooses whichever is smaller between the packet size you requested and the network's max. packet size.

The only work-around I could find for this is to make sure your buffer is large enough to handle the network's max. packet size.

Also note that I have a real video card here (a RealTek thing) that has a different bug - if you ask for a packet size that's larger than the network's max. packet size the PXE ROM will lock up.

To work around both bugs a the same time; I get the network's max. packet size from PXE's "Get UNDI information" and subtracted the expected size of the TCP/IP and TFTP headers. This gives me a value for "requested packet size" that's large enough to work with buggy iPXE, but small enough to work with that buggy real network card.

To be more specific, this is the code I'm using to determine a "safe" packet size to request:

Code:
TFTP_getMaxPacketSize:
     push eax

     cmp dword [PXENVplusStructure],"PXEN"        ;Was the PXENV+ found?
     jne .l1                                      ; no, assume it's so new that PXENV+ stopped existing
     cmp word [PXENVplusStructure+6],0x0201       ;Is it older than version 2.1?
     jb .use512                                   ; yes, use 512 byte packets

.l1:
     cmp dword [PXEstructure],"!PXE"              ;Was the !PXE found?
     jne .use512                                  ; no, shouldn't be possible so be cautious

     movzx eax,word [UNDI_getInfoRequest.MTU]     ;eax = MTU
     sub eax,2*2+4*2+5*4+32                       ;Subtract TFTP header size, UPD header size, IP header size and 16 bytes of "just in case" from MTU
     jc .use512
;     and eax,0xFFFFFFFC                          ;Make it a multiple of 4

     cmp eax,512                                  ;Is it smaller than 512 bytes?
     jb .use512                                   ; yes, that's not sane (TFTP must support 512 or more)
     cmp eax,4096                                 ;Is it larger than 1 page (causes problems with memory management)?
     jb .l2                                       ; no
     mov eax,4096                                 ; yes, limit it to 4 KiB
.l2:
     mov word [packetSize],ax
     pop eax
     ret

.use512:
     mov word [packetSize],512
     pop eax
     ret



Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: [Bochs] PXE TFTP Issues
PostPosted: Sun Oct 25, 2015 1:07 pm 
Offline
Member
Member
User avatar

Joined: Fri Jan 16, 2009 8:34 pm
Posts: 284
Location: Louisiana, USA
@Brendan

I have attempted your suggestion, and it only seems to make it worse - Bochs has the same result, but the 2 PCs I tried on no longer function.

Just to make sure I am doing this right we are talking about:
Code:
PXE_UNDI_GET_INFORMATION equ 0x000C

UNDI_GET_INFORMATION:
   mov di, PXENV_UNDI_GET_INFORMATION_t
   mov bx, PXE_UNDI_GET_INFORMATION
   call usePXE
   test ax, ax
   mov ax, WORD [PXENV_UNDI_GET_INFORMATION_t.Status]
   jnz TFTP_ERROR
   ret

PXENV_UNDI_GET_INFORMATION_t:
   .Status dw 0
   .BaseIo dw 0
   .IntNumber dw 0
   .MaxTranUnit dw 0
   .HwType dw 0
   .HwAddrLen dw 0
   .CurrentNodeAddress times 16 db 0
   .PermNodeAddress times 16 db 0
   .Segment dw 0
   .RxBufCt dw 0
   .TxBufCt dw 0
   .Pad times 16 dd 0 ; This is in the middle of my code ATM lets not destroy anything
PXENV_UNDI_GET_INFORMATION_t.MaxTranUnit on Bochs and the 2 PCs seems to be just a few bytes from 0x05E0 (each only different by a max of 0x20)

Here is the actual values placed into the structure with Bochs:
Code:
0x0000
0xC020
0x0000
0x05DC
0x0001
0x0006
0xC4B0 0x0135 0x0302 0x0000 0x0000 0x0000 0x0000 0x0000
0xC4B0 0x0135 0x0302 0x0000 0x0000 0x0000 0x0000 0x0000
0x0000
0x0001
0x0001

P.S. Yes, I used your math function after getting the value, then I tried my own - rounding down to the nearest 0x0200 Bytes - Bochs just dies as always, and the two test PCs fail because the packet is still 0x0200 while requesting 0x0400 and no negotiated packet size is returned from TFTP_OPEN.

And just for the heck of it I set packet size to 0x1000 and 0xFFFF just to see what Bochs does, it is the same exact thing.



So, for now I'm going to comment my code (and add the newly learned movzx instruction instead of xor then mov...), fix-up a HDD boot loader (again), and say screw emulating PXE for now, lol.
As long as I can execute my Kernel in Bochs I'll be "happy"...
But, I am still open for suggestions.

_________________
BOS Source Thanks to GitHub
BOS Expanded Commentary
Both under active development!
Sortie wrote:
  • Don't play the role of an operating systems developer, be one.
  • Be truly afraid of undefined [behavior].
  • Your operating system should be itself, not fight what it is.


Top
 Profile  
 
 Post subject: Re: [Bochs] PXE TFTP Issues
PostPosted: Sun Oct 25, 2015 11:24 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

BASICFreak wrote:
Nable wrote:
Why do you manually put OperandSize prefix (db 0x66)?
Correct translator (such as FASM and, possibly, NASM) puts them automatically according to operands and assumed CPU mode that you specify using "bits" directive.

What's this:
Code:
      db 0x66
         xor ecx, ecx
      mov cx, ax
?
Did you want to do this:
Code:
movzx ecx, ax
?

Your code is strange for sure.

The reason is simple, nasm assumes I want "xor cx, cx" and will not place the opcode 66h and unless I did "mov ecx, DWORD 0" which without DWORD nasm will assume "mov cx, 0" without the opcode.


That's extremely unlikely. Far more likely is that the assembler did exactly what it should, but you forgot to tell the disassembler the correct code size so it looked wrong when you disassembled it. Note that if you mix 32-bit code and 16-bit code it's impossible to tell the disassembler which parts use which code size and something has to be disassembled incorrectly.

BASICFreak wrote:
Either way the math still makes ecx = (0x00000000 | ax)


The override means that the CPU sees your "0x66 xor ecx,ecx" and treats it as "xor cx,cx"; and the highest bits of ECX still contain whatever they did before.

BASICFreak wrote:
I have attempted your suggestion, and it only seems to make it worse - Bochs has the same result, but the 2 PCs I tried on no longer function.

Just to make sure I am doing this right we are talking about:
Code:
PXE_UNDI_GET_INFORMATION equ 0x000C

UNDI_GET_INFORMATION:
   mov di, PXENV_UNDI_GET_INFORMATION_t
   mov bx, PXE_UNDI_GET_INFORMATION
   call usePXE
   test ax, ax
   mov ax, WORD [PXENV_UNDI_GET_INFORMATION_t.Status]
   jnz TFTP_ERROR
   ret

PXENV_UNDI_GET_INFORMATION_t:
   .Status dw 0
   .BaseIo dw 0
   .IntNumber dw 0
   .MaxTranUnit dw 0
   .HwType dw 0
   .HwAddrLen dw 0
   .CurrentNodeAddress times 16 db 0
   .PermNodeAddress times 16 db 0
   .Segment dw 0
   .RxBufCt dw 0
   .TxBufCt dw 0
   .Pad times 16 dd 0 ; This is in the middle of my code ATM lets not destroy anything
PXENV_UNDI_GET_INFORMATION_t.MaxTranUnit on Bochs and the 2 PCs seems to be just a few bytes from 0x05E0 (each only different by a max of 0x20)


That's likely to be right. The MTU for ethernet is typically 1500 bytes, which means you're expecting the value 0x05DC in most cases.

BASICFreak wrote:
P.S. Yes, I used your math function after getting the value, then I tried my own - rounding down to the nearest 0x0200 Bytes - Bochs just dies as always, and the two test PCs fail because the packet is still 0x0200 while requesting 0x0400 and no negotiated packet size is returned from TFTP_OPEN.


That doesn't make sense. If the two test PCs give you 512 byte packets when you ask for larger (which is perfectly legal - e.g. the TFTP server might only support 512 byte packets even though the network itself handles larger packets); then you'd end up using 512 byte packets, which is what you were using before, which worked before.

Also note that the TFTP protocol isn't very efficient (e.g. the server spends half its time waiting for an "ACK" from the client instead of sending data; and smaller packets mean more data packets and more ACKs which makes it worse, and also more overhead in the form of UDP/IP headers). Basically, for performance, you want to be using the largest packet size you can (which might be 512 bytes for crusty old TFTP servers that don't support the packet size negotiation that was added to TFTP protocol in 1998, but should be more like 1400 bytes for a modern server on an ethernet network).

Do you know why Bochs dies? Does it try to execute an illegal opcode, or consume more stack space than you gave it and trash something, or wait forever for a packet that never arrives, or... ?


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: [Bochs] PXE TFTP Issues
PostPosted: Mon Oct 26, 2015 12:08 am 
Offline
Member
Member
User avatar

Joined: Fri Jan 16, 2009 8:34 pm
Posts: 284
Location: Louisiana, USA
Quote:
That's extremely unlikely. Far more likely is that the assembler did exactly what it should, but you forgot to tell the disassembler the correct code size so it looked wrong when you disassembled it. Note that if you mix 32-bit code and 16-bit code it's impossible to tell the disassembler which parts use which code size and something has to be disassembled incorrectly.
Well tell that to the CPU which triple faults without some of them... I only put them when it had to go there. (pushing F4 on bochs (GUI Debugger) fixes the disassembler output - based on CPU mode, just an FYI) Plus they are not in mode switches just accessing the extended registers in 16-bit code, the only time I do a mode switch is memset, memcpy, and jumping to kernel. And I can watch the stack miss the extended part of lets say eax with push eax vs db 0x66 push eax.

But the movzx r/32, r/16 works fine, and saved me a total of 48 bytes :lol: <- wish I knew about that one sooner... I might even get FAT32 and FAT16 in the same boot sector...

(only time I use a disassembler other than Bochs is with ELFs)

Quote:
That doesn't make sense. If the two test PCs give you 512 byte packets when you ask for larger (which is perfectly legal - e.g. the TFTP server might only support 512 byte packets even though the network itself handles larger packets); then you'd end up using 512 byte packets, which is what you were using before, which worked before.
Yes, and it would work if I wasn't checking for packets smaller than (WORD) PXENV_TFTP_OPEN_t.PacketSize to find EOF

(it's nearly 1AM here, I'll attempt to see what happens if I just read again - hopefully it will return a 0 size and I'll go that way. Tomorrow that is, just before checking here I closed down my Editor and terminal...)

Quote:
Also note that the TFTP protocol isn't very efficient (e.g. the server spends half its time waiting for an "ACK" from the client instead of sending data; and smaller packets mean more data packets and more ACKs which makes it worse, and also more overhead in the form of UDP/IP headers). Basically, for performance, you want to be using the largest packet size you can (which might be 512 bytes for crusty old TFTP servers that don't support the packet size negotiation that was added to TFTP protocol in 1998, but should be more like 1400 bytes for a modern server on an ethernet network).
Yes, though I'm not worried about the time in the boot loader - I cannot even see it once it starts, in fact I pause it waiting on a key for debug purposes (otherwise if something messed up I'd triple fault jumping to kernel)

Quote:
Do you know why Bochs dies? Does it try to execute an illegal opcode, or consume more stack space than you gave it and trash something, or wait forever for a packet that never arrives, or... ?
It doesn't die as in Bochs dying, it dies because it hits a hlt loop

PXE returns ax = 1 and status = 0x0006 (which I then just jump to Error Handler and die [hlt loop])



But in the mean time, I have only 2 more functions before finishing my FAT16 loader - it's way (way, way) more stable. And the MBR and VBR is complete (again).

_________________
BOS Source Thanks to GitHub
BOS Expanded Commentary
Both under active development!
Sortie wrote:
  • Don't play the role of an operating systems developer, be one.
  • Be truly afraid of undefined [behavior].
  • Your operating system should be itself, not fight what it is.


Top
 Profile  
 
 Post subject: Re: [Bochs] PXE TFTP Issues
PostPosted: Mon Oct 26, 2015 1:44 am 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

BASICFreak wrote:
Quote:
That's extremely unlikely. Far more likely is that the assembler did exactly what it should, but you forgot to tell the disassembler the correct code size so it looked wrong when you disassembled it. Note that if you mix 32-bit code and 16-bit code it's impossible to tell the disassembler which parts use which code size and something has to be disassembled incorrectly.
Well tell that to the CPU which triple faults without some of them... I only put them when it had to go there. (pushing F4 on bochs (GUI Debugger) fixes the disassembler output - based on CPU mode, just an FYI) Plus they are not in mode switches just accessing the extended registers in 16-bit code, the only time I do a mode switch is memset, memcpy, and jumping to kernel. And I can watch the stack miss the extended part of lets say eax with push eax vs db 0x66 push eax.


Is it possible that you've got a "bits 32" somewhere (e.g. at the start of your memcpy code) and forgot to switch back to "bits 16" after; causing whatever source code comes next (which can be a completely different file if you're using "%include") to be assembled with the wrong size?

For example:

Code:
     bits 16

     %include "foo"         ;All code in "foo" uses 16 bits
     %include "memcpy"      ;This file has a "bits 32" in it somewhere
     %include "bar"         ;All code in "foo" uses 32 bits because that's what "memcpy" left behind.


BASICFreak wrote:
Quote:
That doesn't make sense. If the two test PCs give you 512 byte packets when you ask for larger (which is perfectly legal - e.g. the TFTP server might only support 512 byte packets even though the network itself handles larger packets); then you'd end up using 512 byte packets, which is what you were using before, which worked before.
Yes, and it would work if I wasn't checking for packets smaller than (WORD) PXENV_TFTP_OPEN_t.PacketSize to find EOF.


That doesn't make too much sense either. Before calling PXE's "TFTP open" function you put a requested size in "PXENV_TFTP_OPEN_t.PacketSize", then PXE's "TFTP open" function determines the actual packet size it wants to let you have and replaces the value in the "PXENV_TFTP_OPEN_t.PacketSize" to tell you what you can have. After that you use the actual packet size that PXE let you have for everything (including for determining EOF) and not the requested packet size.

If you request 4096-byte packets and get 512-byte packets, or if you request 512-byte packets and get 512-byte packets; you'd be using 512-byte packets for the purpose of EOF.

If you request 4096-byte packets and get 1024-byte packets; then you must use 1024-byte packets for the purpose of EOF (and not 4096-byte packets or 512-byte packets).

If it doesn't work, then either your code is buggy or the PXE ROM is buggy.

Note that there's always a possibility that the last packet will fit exactly (e.g. 512-bytes per packet and the file is 5120 bytes, where the last/tenth packet has 512 bytes of data); and in that case you will end up trying to get an extra packet and should end up with a packet of zero bytes.

BASICFreak wrote:
Quote:
Also note that the TFTP protocol isn't very efficient (e.g. the server spends half its time waiting for an "ACK" from the client instead of sending data; and smaller packets mean more data packets and more ACKs which makes it worse, and also more overhead in the form of UDP/IP headers). Basically, for performance, you want to be using the largest packet size you can (which might be 512 bytes for crusty old TFTP servers that don't support the packet size negotiation that was added to TFTP protocol in 1998, but should be more like 1400 bytes for a modern server on an ethernet network).
Yes, though I'm not worried about the time in the boot loader - I cannot even see it once it starts, in fact I pause it waiting on a key for debug purposes (otherwise if something messed up I'd triple fault jumping to kernel)


Initially (when an OS project is just beginning) the files you download via. TFTP are tiny and download speed isn't a problem. Eventually (when an OS project grows and starts to become useful) the file/s grow and download speed becomes a massive problem. Try using your code to download a 123 MiB file (and then compare how long it takes to something like NTFS or FTP) and you'll see what I mean..

BASICFreak wrote:
Quote:
Do you know why Bochs dies? Does it try to execute an illegal opcode, or consume more stack space than you gave it and trash something, or wait forever for a packet that never arrives, or... ?
It doesn't die as in Bochs dying, it dies because it hits a hlt loop

PXE returns ax = 1 and status = 0x0006 (which I then just jump to Error Handler and die [hlt loop])


Ah - I understand now. "TFTP open" returns "status = 0x0006/PXENV_STATUS_OUT_OF_RESOURCES" without opening anything (and probably without changing the packet size). In that case something is probably very wrong - e.g. you've already got a connection active, or you've unloaded UNDI, or you're seeing "undefined behaviour" from bizarre operand size shenanigans and/or bugs, or the stack isn't large enough, or (maybe) the virtual machine is configured for 1 MiB of RAM (and the PXE ROM expects the system has more RAM), or...


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: [Bochs] PXE TFTP Issues
PostPosted: Mon Oct 26, 2015 3:29 pm 
Offline
Member
Member
User avatar

Joined: Fri Jan 16, 2009 8:34 pm
Posts: 284
Location: Louisiana, USA
BTW, all the TFTP code no longer has any modification opcode - movzx r/32, r/16 works better.

And 3/5 PCs actually negotiate the packet size, the other two (netbook [atom] and P4ht - both HP systems) do not - which are the ones I test on as they are the most finicky. (I do not recall the actual NIC manufactures - but they are onboard NICs)

My stack size when calling TFTP open is about 0x7000 (located at 0x0000:0xFFF0 to start)

I've attempted to close any active connections when I first started this thread, It still failed the same way - but HW failed too, as you are not supposed to call a Close function without an open connection.

Bochs is setup with:
Code:
config_interface: textconfig
display_library: sdl, options="gui_debug"
memory: host=256, guest=512
romimage: file="/usr/share/bochs/BIOS-bochs-latest"
vgaromimage: file="/usr/share/bochs/VGABIOS-stdvga"

pci: enabled=1, chipset=i440fx, slot1=pcivga, slot2=ne2k
ne2k: enabled=1, mac=B0:C4:35:01:02:03, ethmod=linux, ethdev=br0, bootrom="./iPXE_10ec8029.rom"
boot: network, disk

... ATA and FDD info ...

vga: extension=vbe, update_freq=60
cpu: count=1, ips=1000000, model=bx_generic, reset_on_triple_fault=1, cpuid_limit_winnt=0, ignore_bad_msrs=1, mwait_is_nop=0
cpuid: family=6, model=0x03, stepping=3, mmx=1, apic=xapic, simd=sse2, sse4a=0, sep=1, aes=0, xsave=0, xsaveopt=0, movbe=0, adx=0, smep=0, bmi=0, xop=0, tbm=0, fma4=0, vmx=1, x86_64=1, 1g_pages=0, pcid=1, fsgsbase=0, mwait=1
cpuid: vendor_string="GenuineIntel"
cpuid: brand_string="              Intel(R) Pentium(R) 4 CPU        "

... Debug, COM, KB, Mouse, and LPT info ...


Also, if I were to load lets say 128MB - I would (90% chance) make a "stage three" loader to use sftp and use tftp only to load that stage. But, for now, I load less than 2MB - though I see it getting to 16-32MB soon (as I get drivers and functionality)

I'm off to write those two remaining FAT functions now... (search sub-dir and load file high-mem)




I really hate the timezone differences between me and most the ([more] experienced) users here... (CDT) UTC-5 and (CST) UTC-6 (depending on daylight savings which ends Oct 1...)

EDIT: I concede defeat on the modification opcodes, well most of them, only 2 are REQUIRED to be there (and it happened to be the two 0x67s - which are in memcpy and memset)

Also neither memcpy or memset switch to 32-bit mode, only just before jumping to the kernel do I switch to a 32-bit code and 32-bit data segment.

Code:
[bits 16]

;----------------------------------------------------
;                       memset
;   INPUT: EDI - destination ECX - count AL - value
;----------------------------------------------------
memset:
   cli
   push ds
   push es
   mov  ebx, cr0         ; switch to pmode by
   push ebx
   or bl,1               ; set pmode bit
   mov  cr0, ebx

   mov bx, 10h            ; set 32-bit Data GDT ent
   mov ds, bx
   mov es, bx
   jmp 8:.pmode         ; set 16-bit Code GDT ent
   .pmode:
      db 0x67
         rep stosb         ; copy image to its protected mode address
      pop ebx
      mov  cr0, ebx         ; Go back to RM
      jmp 0x0:.done
   .done:
      pop es            ; load real mode Data Segments
      pop ds
      sti
   ret

;----------------------------------------------------
;                       memcpy
;  INPUT: ESI - source EDI - destination ECX - count
;   NOTE: This counts in BYTES (1 Bytes per count)
;----------------------------------------------------
memcpy:
   cli
   push ds               ; save real mode Data Segments
   push es
   mov  ebx, cr0         ; switch to pmode by
   push ebx
   or bl,1               ; setting pmode bit
   mov  cr0, ebx
   mov bx, 10h            ; set 32-bit Data GDT ent
   mov ds, bx
   mov es, bx
   jmp 8:(.pmode)         ; set 16-bit Code GDT ent
   .pmode:
        db 0x67
         rep   movsb         ; copy image to its protected mode address
      pop ebx
      mov  cr0, ebx
      jmp 0x0:.done
   .done:
      pop es            ; load real mode Data Segments
      pop ds
      sti
   ret

_________________
BOS Source Thanks to GitHub
BOS Expanded Commentary
Both under active development!
Sortie wrote:
  • Don't play the role of an operating systems developer, be one.
  • Be truly afraid of undefined [behavior].
  • Your operating system should be itself, not fight what it is.


Top
 Profile  
 
 Post subject: Re: [Bochs] PXE TFTP Issues
PostPosted: Mon Oct 26, 2015 7:56 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5099
BASICFreak wrote:
EDIT: I concede defeat on the modification opcodes, well most of them, only 2 are REQUIRED to be there (and it happened to be the two 0x67s - which are in memcpy and memset)

You are supposed to use the a32 prefix in this situation.


Top
 Profile  
 
 Post subject: Re: [Bochs] PXE TFTP Issues
PostPosted: Mon Oct 26, 2015 8:26 pm 
Offline
Member
Member
User avatar

Joined: Fri Jan 16, 2009 8:34 pm
Posts: 284
Location: Louisiana, USA
Octocontrabass wrote:
BASICFreak wrote:
EDIT: I concede defeat on the modification opcodes, well most of them, only 2 are REQUIRED to be there (and it happened to be the two 0x67s - which are in memcpy and memset)

You are supposed to use the a32 prefix in this situation.

Thanks :wink:, worked like a charm (same result in hex, but cleaner for sure)

Ok, now there are NO MORE silly 0x66 or 0x67s in my code.

Now we can concentrate solely on the PXE issue. :?

_________________
BOS Source Thanks to GitHub
BOS Expanded Commentary
Both under active development!
Sortie wrote:
  • Don't play the role of an operating systems developer, be one.
  • Be truly afraid of undefined [behavior].
  • Your operating system should be itself, not fight what it is.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 13 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 37 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group