Joel Fernandes
2025-Nov-10 15:16 UTC
[PATCH v2 08/12] nova-core: sequencer: Add register opcodes
really +Steve Rostedt this time. On 11/10/2025 10:16 AM, Joel Fernandes wrote:> On 11/5/2025 6:19 PM, Timur Tabi wrote: >> On Wed, 2025-11-05 at 13:55 -0800, John Hubbard wrote: >>>> #define nvdev_trace(d,f,a...) nvdev_printk((d), TRACE,?? info, f, ##a) >>>> #define nvdev_spam(d,f,a...)? nvdev_printk((d),? SPAM,??? dbg, f, ##a) >>> >>> ...and those are unusable, unfortunately. I've tried. >> >> This works great for me: >> >> modprobe nouveau dyndbg="+p" modeset=1 debug="gsp=spam" config=NvGspRm=1 >> >> I get all sequencer messages when I boot with these options. >> >>> ftrace/bpftrace, maybe those are the real way to "trace"...or something >>> other than this. >> >> You could say the same thing about most dev_dbg() statements. >> >> I agree that dev_dbg for sequencer commands is excessive, and that implementing new debug levels >> just to get sequencer prints is also excessive. But Nouveau implement nvkm_trace for a reason. And >> we all know that because of ? in Rust, NovaCore does a terrible job at telling us where an error >> actually occurred. So there is a lot of room for improvement. > > IMO, the best way to do this is the tracing subsystem. It is the lowest overhead > runtime kernel logging system that I know off, lockless, independent of the > serial console etc, next to no runtime overhead when off, etc. > > I recommend we use the tracing subsystem for "trace" and even "spam" level > logging levels for Nova. The brave souls can always ask the tracing subsystem to > also spam to kernel logs if they so wish. > > ++ Tracing Czar Steven Rostedt as well. Steve, Nova is a new modern Nvidia GPU > driver. > > I guess we have to decide how to do this - what kind of tracepoints do we need > for Nova. One use case that just came up is RPC message buffer dumps for > debugging communication with the firmware. > > thanks, > > - Joel >
Steven Rostedt
2025-Nov-10 16:59 UTC
[PATCH v2 08/12] nova-core: sequencer: Add register opcodes
On Mon, 10 Nov 2025 10:16:45 -0500 Joel Fernandes <joelagnelf at nvidia.com> wrote:> > IMO, the best way to do this is the tracing subsystem. It is the lowest overhead > > runtime kernel logging system that I know off, lockless, independent of the > > serial console etc, next to no runtime overhead when off, etc. > > > > I recommend we use the tracing subsystem for "trace" and even "spam" level > > logging levels for Nova. The brave souls can always ask the tracing subsystem to > > also spam to kernel logs if they so wish. > > > > ++ Tracing Czar Steven Rostedt as well. Steve, Nova is a new modern Nvidia GPU > > driver.Not sure if there was a question here, but you can enable tracing via the kernel command line and it will record into the ring buffer, and read it out via /sys/kernel/tracing/trace. You could also create instances by the kernel command line that will place events in different ring buffer instances (/sys/kernel/tracing/instances/<instance>/trace). You could even filter the events based on the trace event fields, process ID, CPU, etc. There's also a tp_printk kernel command line option that will turn every trace event into a printk() statement. You need to be careful of what events you enable when doing this, as some events (like locking events) can live lock the system if they are piped to printk(). After boot up, you can turn off the tp_printk with: echo 0 > /proc/sys/kernel/tracepoint_printk -- Steve> > > > I guess we have to decide how to do this - what kind of tracepoints do we need > > for Nova. One use case that just came up is RPC message buffer dumps for > > debugging communication with the firmware.