Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

USB DMX output timing odd behaviour on Windows #1656

Open
basilhussain opened this issue Jan 3, 2025 · 5 comments
Open

USB DMX output timing odd behaviour on Windows #1656

basilhussain opened this issue Jan 3, 2025 · 5 comments

Comments

@basilhussain
Copy link

I have discovered some odd behaviour with regard to timing of DMX output frames when using a USB DMX interface on Windows.

Basically, it seems the simple act of opening and closing the output plug-in configuration window - without adjusting any of the settings - ruins the timing of DMX frames being output using that interface. The break and MAB periods become excessively long (15+ ms each) and QLC+ is no longer able to output full frames of all 512 slots at the required frame frequency because there isn't enough time - the slots get cut off (i.e. <512) by the break of the subsequent frame.

Bizarrely, when this happens, system timer accuracy is reported by the plug-in to be "Good", despite that not actually being the case. In fact, the only time I have managed to get sensible frame timing is when accuracy is reported as "Bad". (This I believe is because under that situation, QLC+ doesn't attempt to do any timing for break or MAB, and just lets them occur as quickly as possible.)

From digging through the QLC+ source code, I can't figure out exactly the root of the problem, but I think it is probably something to do with the following:

  • QThread's usleep implementation on Windows rounds up the given usecs argument to the nearest millisecond. This renders QLC+'s attempt to do microsecond-resolution sleeps (using DMX_MAB of 16 and DMX_BREAK of 110 µs) pointless.
  • The usleep functionality is delegated to Windows' Sleep API function, which also only deals in milliseconds. And, importantly, the minimum resolution of Sleep is reliant on the current resolution of the system timer.
  • Unless a Windows application specifically requests a lower system timer resolution, it can often be as high as 15.625 ms. This probably explains the 15 ms break and MAB timings I have been seeing.
  • If the system timer resolution being 15 ms is the cause of the problem, I am puzzled by how QLC+ determines that timing accuracy is "Good" in that situation. It seems it measures if a 1000 µs usleep actually takes more than 3 ms, and is "Bad" if so. But if it is taking 15 ms, how can it be less than 3 ms? Perhaps the accuracy of QElapsedTimer is also affected by system timer resolution; if so, it would not be surprising if such a test gives flawed results.
  • To avoid the system timer resolution being higher than needed, it's my understanding that Windows applications are supposed to use timeGetDevCaps and timeBeginPeriod/timeEndPeriod to request a certain timer resolution for the life of the application.
  • It seems as though QLC+ has some code somewhere (in MasterTimerPrivate::start) that uses those API functions, but I couldn't figure out under what circumstances they are called. Whenever or wherever it is used, it doesn't seem to have any effect on DMX USB timing.

To Reproduce

Steps to reproduce the behaviour:

  1. Connect an FTDI-based USB DMX interface.
  2. Launch QLC+.
  3. Go to Inputs/Outputs tab. Ensure DMX USB interface is mapped to a Universe.
  4. For that interface, note that "System Timer Accuracy" is reported as "Bad".
  5. Using an oscilloscope or logic analyser, capture the DMX traffic and observe that the frame timing is reasonable (break and MAB are both <200 µs).
  6. With the relevant DMX USB interface selected, click icon button to open the plug-in configuration window.
  7. Do not touch anything - just click 'Close' immediately.
  8. Note that "System Timer Accuracy" is now reported as "Good".
  9. Capture the DMX traffic again. Observe that the frame timing has gone to crap. Break and MAB are now 15+ ms, Frames are incomplete (<512 slots) before being cut off by the break of next frame.

Expected Behaviour

DMX frame timing of break and MAB should be minimal (i.e. approx. 1 ms for each, within the limitations of single-millisecond-resolution timing available on Windows), and frame slots should not be prematurely cut off by the transmission of a subsequent frame.

Screenshots

Logic analyser DMX traffic capture showing bad timing:
QLC+ Bad DMX Timing

Detail of above capture showing cut off frame (only 339 slots transmitted):
QLC+ Bad DMX Timing Cut Off Slots

Desktop

  • OS: Windows 10 22H2 (build 19045.5247)
  • QLC+ Version: 4.13.1
  • DMX Interface: FTDI FT2232H
@GGGss
Copy link

GGGss commented Jan 4, 2025

I recently performed a study to instruct users of a uDMX interface to adopt and tweak their timing settings by buying a uDMX interface.

This led to this forum thread: https://www.qlcplus.org/forum/viewtopic.php?t=17954&hilit=uDMX
I studied the code behind the interface and never would have thought that the code severely lacked the right timing, blaming the processor's thread for not coping with the amount of data sent. DMX is barely 250kbps - this is 80'ies stuff...

What I did see was, indeed, the usleep timers ... I'm not acquainted enough to make the judgement - I only studied the outcome, and ^^ observations reflect almost exactly what I have measured.

I commented @mcallegari asking for the non-existent influence by setting values in the registry. ^^ Observations might strike lightning. I never considered the qt vs API vs sleep deviations or why data output was sluggish.

Link to the DMX timing chart I'm holding (since I had my cat measuring device): https://docs.google.com/spreadsheets/d/1mah0i1ffiByNordS4c7oKktONagDmoJiu1nrE-5VC6Y/edit?usp=sharing
Study the $A$71 topic: Calculation.
In my study, the MaB (Mark after Break) timings were consistent with the OP's observations. They are within the allowed DMX standards, but these timings force the DMX-bus frequency to be drastically lowered.
If the DMX-bus frequency is set to default, in QLC+, DMX frames will not be completed within the standard's specifications, and hence, information will be lost. This results in moderate to significant flickering depending on the devices' firmware attached to the DMX universe.

Sorry, I cannot contribute to the code - I'm more than willing to test the outcome...

@basilhussain
Copy link
Author

Yes, @GGGss, your observations seem to match mine.

But after a little confusion and investigation, I think you have some nomenclature mix-up with the name "uDMX". I am not too familiar with these things, but as far as I can discern, the "uDMX" that QLC+ supports is the Anyma uDMX, which is an open-source DMX interface based on an Atmel AVR microcontroller, whereas cheap interfaces labelled "uDMX" available for purchase online (e.g. AliExpress, eBay) - and what you have - are typically based on FTDI chips. Be aware they work in very different ways!

The 'original' uDMX cannot suffer from frame timing issues like these, as the embedded microcontroller generates the DMX break, MAB, etc. by itself. That is, the timing is not decided by the host computer, but independently by the interface. But with the FTDI-based interfaces, the host computer (i.e. QLC+) is "in the driving seat" so to speak.

So, whatever those registry values you refer to don't apply here - that's for the "uDMX" interface plug-in, but we here with FTDI interfaces are dealing with the "USB DMX" plug-in. The frame frequency is settable within the plug-in config window.

By the way, that was one thing I forgot to mention before: I played with adjusting the frame frequency down to a lower rate, and found that it too has poor timing accuracy. For example, I set it to 20 Hz, but what I actually got was around 16 Hz.

With regard to allowed timings by the standard, I did wonder whether any of this was verging into failing to meet specifications, but after checking the ANSI E1.11 specifications, it's technically okay. The standard specifies no maximum break length (only minimum), and MAB has a very high maximum length of 1 second. Zero MBB time is also allowed. No minimum number of slots is specified either, so technically outputting all 512 is at the controller's discretion.

But, as we all know, something that is technically within specs may equally be of no reliable use in the real world. 😄

@basilhussain
Copy link
Author

basilhussain commented Jan 4, 2025

I have been doing some more investigation.

It is my understanding that in Windows editions before Windows 10 version 2004, the system timer resolution will globally be whatever the smallest value requested by any application or process is - e.g. if Process A requests 8 ms, and Process B requests 1 ms, everyone gets 1 ms timer resolution. If no-one requests anything, the default is 16 ms.

However, in Windows 10 version 2004 (released May 2020) and onwards (presumably including Windows 11) the behaviour was changed. Now timer resolution is essentially per-process. If a process requests a certain resolution, it may still end up with a lower value that some other process has already requested. But if a process makes no request (i.e. never calls timeBeginPeriod), Windows will synthesise a timer resolution approximating the default 16 ms, even if the actual resolution is lower. More info in this blog post.

There's also one other wrinkle: with Windows 11, whenever a process is minimised (or otherwise not visible to the user), Windows will also temporarily downgrade the system timer resolution for that process, even if it has requested a lower resolution, as part of the default power-saving scheme. If applications want to opt-out of this they need to call the SetProcessInformation API function.

The change in behaviour described above explains why I (on Win 10 22H2) wasn't having any luck forcing a lower system timer resolution. I happened to discover that running a game in the background lowered it to 1 ms, and running the Microsoft SysInternals ClockRes utility showed as much. But yet QLC+ DMX timing was still unaffected and bad.

So, I really wanted to know how the code that calls Windows API functions timeBeginPeriod/timeEndPeriod in MasterTimerPrivate is invoked, so I can perhaps get QLC+ to 'ask' Windows for a lower system timer resolution. It seems that the parent MasterTimer class is actually used all over QLC+, so the likelihood of this code being invoked to me seems high.

After reading the code more thoroughly I note that what MasterTimePrivate::start does is as follows:

  1. Calls timeGetDevCaps to query the minimum and maximum possible system timer resolution (typically 0.5 and 16.625 ms respectively).
  2. Calls timeBeginPeriod with a value calculated as:
    • If the MasterTimer instance's tick parameter is greater than the minimum possible, use tick;
    • But if it is greater than the maximum possible, use the max.

Where does the tick parameter come from? It defaults to 20 ms, unless the mastertimer/frequency registry parameter for QLC+ has been set. Aha!

So I created that registry key and value, setting it to 125 (for 125 Hz). And it had an effect! Instead of the DMX timing going to crap after the plug-in config window is opened-closed, I got break and MAB timings of ~2 ms, full 512 slot frames, and proper break-to-break frequency of ~30 Hz! A bit weird, though, given I was expecting it to have been 1000/125=8 ms that timeBeginPeriod would have been passed.

However, after some experimentation, I am now more confused than ever... 😕

Multiple different values of mastertimer/frequency config always seems to result in either 2 ms break/MAB, or bad 16 ms. Tried 125 Hz, 75 Hz: good 2 ms timing. Tried 60 Hz, 50 Hz: bad 16 ms timing.

I also discovered that the Windows built-in powercfg utility can report on what is happening with the system timer resolution. Running powercfg /energy will collect data for a 60-sec period and write out an HTML report file, part of which says what applications did what with the timer. Confusingly, launching QLC+ and playing around never gets qlcplus.exe included in the powercfg report as having altered system timer resolution!

This is an example of a report where an app (a game) does request different system timer resolution:
powercfg report

So if QLC+ never triggers inclusion in the report, it appears never to actually be calling timeBeginPeriod, but yet somehow the registry setting has an effect and something is changed with how Windows is running the system timer.

@mcallegari
Copy link
Owner

Sorry TL;DR
What do you suggest that I do?
Please note that Master timer and DMX USB frequencies must be independent

@basilhussain
Copy link
Author

After some more consideration, I think we can narrow it down to two main problems:

1. The system timer accuracy measurement by the DMX USB plug-in is inconsistent

When you first launch QLC+ (or first map the plug-in), it correctly determines that accuracy is "bad" (because a 1000 µs sleep actually takes >3 ms), but when you open and then close the plug-in's config window, it somehow does not do this measurement properly (or maybe at all) and incorrectly determines accuracy to be "good". I believe this to be a bug. The effect is then that when it mistakenly thinks accuracy is "good", the plug-in attempts to perform proper break/MAB timing, but this actually has a detrimental effect and results in crappy DMX timing and truncated frames, all due to inadequate system timer resolution.

I have tried to determine why this bug is occurring, but due to my unfamiliarity I can't follow the code through what happens when the plug-in's config window is opened and closed. If I had to guess, I'd say it's something to do with the plug-in being re-initialised in a different way to when the app first launches.

2. A futile attempt by MasterTimerPrivate::start to adjust the system timer resolution could be aiding DMX timing, but doesn't

When the mastertimer/frequency registry setting is either not present (and thus defaulting to 50 Hz, 20 ms period), or is set to anything <66 Hz (>15 ms period). this results in no change to the system timer resolution. From experimentation I have determined that passing of any value greater than the default system timer resolution (15.625 ms) to the timeBeginPeriod API function is ignored by Windows. As I mentioned before, only when you set a higher mastertimer/frequency value (e.g. 75 Hz) does the system timer resolution get altered, with the side-effect that the DMX USB plug-in is then able to perform better output timing.

(By the way, I discovered the reason why I wasn't previously seeing qlcplus.exe in the powercfg report is that it only catches system timer resolution changes already in effect, and doesn't catch new ones. I was starting powercfg, then launching QLC+, when instead I should have been doing the opposite.)

What do you suggest that I do?

In light of the above, here are my suggestions:

  • When starting the DMX USB plug-in (i.e. in the run method) under Windows, unconditionally use timeGetDevCaps and timeBeginPeriod to request a system timer resolution of 1 ms. Also call timeEndPeriod when plug-in is stopped (i.e. in stop method). Do this also in any other DMX interface plug-in that does timing via QThread::usleep et al. According to the MS documentation, it is perfectly fine for an application to call timeBeginPeriod multiple times. This will ensure that DMX plug-ins get the appropriate system timer resolution they need, regardless of whatever else is going on.
  • In the Windows implementation of MasterTimerPrivate::start, don't pass the parent MasterTimer::tick value to timeBeginPeriod, but instead pass a fixed value of 1 ms. I believe the current behaviour is misguided. The purpose of this API function is to request a specific resolution of system timer, not to tell Windows what period you will be running timers at. To explain another way: the present behaviour is effectively telling Windows "Just give me a timer that's accurate to within plus-or-minus 20 ms". If you're running a timer at a 20 ms period (50 Hz), you want those intervals to occur exactly every 20 ms, yes? - i.e. 20±1 ms, and not just 20±20 ms. I believe this will have the benefit of improved timing accuracy across the entire application under Windows.
  • I think it would also be prudent to add some 'future-proofing' for Windows 11 and implement a call to the SetProcessInformation API function at application launch to opt-out of reduced system timer resolution while app is minimised, non-visible, etc. I'm sure users would care more about accurate timing than laptop battery life. 😄

Please note that Master timer and DMX USB frequencies must be independent

I don't think I suggested or implied that they should be. Sorry if I gave that impression. I was just trying to describe how increasing the master timer frequency currently has the side effect of correcting DMX USB timing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants