OSDev.org

Does anybody ever made custom gdb stubs for non standard interface hardware?

I have not well known ARM9 SoC based on the arm926 core.
The hardware I have has no JTAG routed, and no working Ethernet (uses Microchip's RMII extender I must to debug).

The single serial port must work with the RS485 interface, so in the result, the only free interfaces I can use for debugging is CAN or generic SPI.

I made a bare metal arm-none-eabi GNU toolchain build for arm926 (with newlib porting in progress), but to be more close to the main task I must solve, how should I write custom gdb stub for debugging w/o use of JTAG features?

You need three things:

1) On the device side, a typical gdb stub which generates the correct responses to messages, e.g. return the current register contents in response to the 'g' command etc. GDB provides typical examples of these for common architectures in the gdb/stubs directory within its source tree (interestingly ARM is not there, however).

2) Also on the device side, a way of letting the gdb stub communicate with the system running gdb. For the stubs listed above, this is via the put/getDebugChar() functions, which you would typically wire up to a serial port or network address or SPI etc

3) On the gdb side, some way for GDB to communicate using the particular protocol you are using. I don't believe it has a built in SPI function, so you may want to look into a SPI->TTY driver or SPI->network, which it can use with the target remote command.

The benefit of splitting it this way means you can easily test each bit in isolation.

For reference, I created a custom gdb stub here which combines parts 1) and 2) above, however aside from giving pointers as to how the stub should run it is probably of little direct use to you because it targets x86_64, uses serial ports and is written in C#.

Regards,
John.