Time spent in ticker event handlers / new use for housekeeping task
A question to the FreeRTOS gurus:
How much time may we spend in a ticker event handler, or generally a software timer callback, without risking system stability?
The FreeRTOS manual explicitly warns about using any kind of blocking calls within the timer service task, as all software timer callbacks are executed in that context:
https://freertos.org/RTOS-software-timer.html
/"//Timer callback functions execute in the context of the timer service task. It is therefore// /*/essential/*/ //that timer callback functions never
attempt to block. For example, a timer callback function must not call vTaskDelay(), vTaskDelayUntil(), or specify a non zero block time when accessing a
queue or a semaphore.//"/The timer service queue currently has 10 slots. That's tunable, but I'm also worried about other system components possibly relying on regular timer periods of higher frequency.
My specific issue: I'm producing the notifications from my ticker1 handler. I now found out a simple standard info notification needs already around 10 ms, and my battery status update (15 data messages) needs 60-70 ms. I thought these would run faster, not sure where the time is spent. I'm using the command notifications, nice to use but with quite some overhead of course.
So if for example the wifi or bluetooth stack needs a 50 ms timer, that may already be a problem. But do system components use software timers for time critical tasks?
The solution would be moving the notification generator to a separate task. A new task would need RAM, but there's also the housekeeping task, which could generally be made available for such needs. It's basically idle after the init process and already has a large stack. I think about adding a callback execution queue to it, so my ticker handler would simply queue the notification generator call there.
Thanks, Michael
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
Michael,
Yes, this is nasty and really not the correct way of doing things. The housekeeping task is wasted, and moving the ticker signals into that would make much more sense.
I’ll handle this. Shouldn’t take me long, and I can commit in the next hour or so.
Regards, Mark.
On 29 Mar 2018, at 2:50 AM, Michael Balzer <dexter@expeedo.de> wrote:
A question to the FreeRTOS gurus:
How much time may we spend in a ticker event handler, or generally a software timer callback, without risking system stability?
The FreeRTOS manual explicitly warns about using any kind of blocking calls within the timer service task, as all software timer callbacks are executed in that context:
https://freertos.org/RTOS-software-timer.html <https://freertos.org/RTOS-software-timer.html> "Timer callback functions execute in the context of the timer service task. It is therefore essential that timer callback functions never attempt to block. For example, a timer callback function must not call vTaskDelay(), vTaskDelayUntil(), or specify a non zero block time when accessing a queue or a semaphore." The timer service queue currently has 10 slots. That's tunable, but I'm also worried about other system components possibly relying on regular timer periods of higher frequency.
My specific issue: I'm producing the notifications from my ticker1 handler. I now found out a simple standard info notification needs already around 10 ms, and my battery status update (15 data messages) needs 60-70 ms. I thought these would run faster, not sure where the time is spent. I'm using the command notifications, nice to use but with quite some overhead of course.
So if for example the wifi or bluetooth stack needs a 50 ms timer, that may already be a problem. But do system components use software timers for time critical tasks?
The solution would be moving the notification generator to a separate task. A new task would need RAM, but there's also the housekeeping task, which could generally be made available for such needs. It's basically idle after the init process and already has a large stack. I think about adding a callback execution queue to it, so my ticker handler would simply queue the notification generator call there.
Thanks, Michael
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Done, and committed. Seems ok.
Regards, Mark.
On 29 Mar 2018, at 8:39 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Michael,
Yes, this is nasty and really not the correct way of doing things. The housekeeping task is wasted, and moving the ticker signals into that would make much more sense.
I’ll handle this. Shouldn’t take me long, and I can commit in the next hour or so.
Regards, Mark.
On 29 Mar 2018, at 2:50 AM, Michael Balzer <dexter@expeedo.de <mailto:dexter@expeedo.de>> wrote:
A question to the FreeRTOS gurus:
How much time may we spend in a ticker event handler, or generally a software timer callback, without risking system stability?
The FreeRTOS manual explicitly warns about using any kind of blocking calls within the timer service task, as all software timer callbacks are executed in that context:
https://freertos.org/RTOS-software-timer.html <https://freertos.org/RTOS-software-timer.html> "Timer callback functions execute in the context of the timer service task. It is therefore essential that timer callback functions never attempt to block. For example, a timer callback function must not call vTaskDelay(), vTaskDelayUntil(), or specify a non zero block time when accessing a queue or a semaphore." The timer service queue currently has 10 slots. That's tunable, but I'm also worried about other system components possibly relying on regular timer periods of higher frequency.
My specific issue: I'm producing the notifications from my ticker1 handler. I now found out a simple standard info notification needs already around 10 ms, and my battery status update (15 data messages) needs 60-70 ms. I thought these would run faster, not sure where the time is spent. I'm using the command notifications, nice to use but with quite some overhead of course.
So if for example the wifi or bluetooth stack needs a 50 ms timer, that may already be a problem. But do system components use software timers for time critical tasks?
The solution would be moving the notification generator to a separate task. A new task would need RAM, but there's also the housekeeping task, which could generally be made available for such needs. It's basically idle after the init process and already has a large stack. I think about adding a callback execution queue to it, so my ticker handler would simply queue the notification generator call there.
Thanks, Michael
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Mark,
thanks, that's even better. We can also reduce the timer task stack now. 6K is the current size, I think we should now be able to set this to 1K, maybe 1.5. I'll try running it at 1K and report.
For other ticker users: you may now use blocking calls (i.e. semaphore or queue waits) and do longer processing in ticker event handlers. Up to 1 second total time should be possible now (but not recommended). If you exceed 1 second, the monotonictime will start to jump and lose sync with real time.
Regards, Michael
Am 29.03.2018 um 03:09 schrieb Mark Webb-Johnson:
Done, and committed. Seems ok.
Regards, Mark.
On 29 Mar 2018, at 8:39 AM, Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> wrote:
Michael,
Yes, this is nasty and really not the correct way of doing things. The housekeeping task is wasted, and moving the ticker signals into that would make much more sense.
I’ll handle this. Shouldn’t take me long, and I can commit in the next hour or so.
Regards, Mark.
On 29 Mar 2018, at 2:50 AM, Michael Balzer <dexter@expeedo.de <mailto:dexter@expeedo.de>> wrote:
A question to the FreeRTOS gurus:
How much time may we spend in a ticker event handler, or generally a software timer callback, without risking system stability?
The FreeRTOS manual explicitly warns about using any kind of blocking calls within the timer service task, as all software timer callbacks are executed in that context:
https://freertos.org/RTOS-software-timer.html
/"//Timer callback functions execute in the context of the timer service task. It is therefore// /*/essential/*/ //that timer callback functions never attempt to block. For example, a timer callback function must not call vTaskDelay(), vTaskDelayUntil(), or specify a non zero block time when accessing a queue or a semaphore.//"/The timer service queue currently has 10 slots. That's tunable, but I'm also worried about other system components possibly relying on regular timer periods of higher frequency.
My specific issue: I'm producing the notifications from my ticker1 handler. I now found out a simple standard info notification needs already around 10 ms, and my battery status update (15 data messages) needs 60-70 ms. I thought these would run faster, not sure where the time is spent. I'm using the command notifications, nice to use but with quite some overhead of course.
So if for example the wifi or bluetooth stack needs a 50 ms timer, that may already be a problem. But do system components use software timers for time critical tasks?
The solution would be moving the notification generator to a separate task. A new task would need RAM, but there's also the housekeeping task, which could generally be made available for such needs. It's basically idle after the init process and already has a large stack. I think about adding a callback execution queue to it, so my ticker handler would simply queue the notification generator call there.
Thanks, Michael
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
If you exceed 1 second, the monotonictime will start to jump and lose sync with real time.
There is actually a way around that (increment monotonictime in the timer handler, not in housekeeping task). The ticker.* calls still won’t go, but monotonictime will at least still increase at a reasonably accurate rate.
Regards, Mark.
On 29 Mar 2018, at 11:46 PM, Michael Balzer <dexter@expeedo.de> wrote:
Mark,
thanks, that's even better. We can also reduce the timer task stack now. 6K is the current size, I think we should now be able to set this to 1K, maybe 1.5. I'll try running it at 1K and report.
For other ticker users: you may now use blocking calls (i.e. semaphore or queue waits) and do longer processing in ticker event handlers. Up to 1 second total time should be possible now (but not recommended). If you exceed 1 second, the monotonictime will start to jump and lose sync with real time.
Regards, Michael
Am 29.03.2018 um 03:09 schrieb Mark Webb-Johnson:
Done, and committed. Seems ok.
Regards, Mark.
On 29 Mar 2018, at 8:39 AM, Mark Webb-Johnson <mark@webb-johnson.net <mailto:mark@webb-johnson.net>> wrote:
Michael,
Yes, this is nasty and really not the correct way of doing things. The housekeeping task is wasted, and moving the ticker signals into that would make much more sense.
I’ll handle this. Shouldn’t take me long, and I can commit in the next hour or so.
Regards, Mark.
On 29 Mar 2018, at 2:50 AM, Michael Balzer <dexter@expeedo.de <mailto:dexter@expeedo.de>> wrote:
A question to the FreeRTOS gurus:
How much time may we spend in a ticker event handler, or generally a software timer callback, without risking system stability?
The FreeRTOS manual explicitly warns about using any kind of blocking calls within the timer service task, as all software timer callbacks are executed in that context:
https://freertos.org/RTOS-software-timer.html <https://freertos.org/RTOS-software-timer.html> "Timer callback functions execute in the context of the timer service task. It is therefore essential that timer callback functions never attempt to block. For example, a timer callback function must not call vTaskDelay(), vTaskDelayUntil(), or specify a non zero block time when accessing a queue or a semaphore." The timer service queue currently has 10 slots. That's tunable, but I'm also worried about other system components possibly relying on regular timer periods of higher frequency.
My specific issue: I'm producing the notifications from my ticker1 handler. I now found out a simple standard info notification needs already around 10 ms, and my battery status update (15 data messages) needs 60-70 ms. I thought these would run faster, not sure where the time is spent. I'm using the command notifications, nice to use but with quite some overhead of course.
So if for example the wifi or bluetooth stack needs a 50 ms timer, that may already be a problem. But do system components use software timers for time critical tasks?
The solution would be moving the notification generator to a separate task. A new task would need RAM, but there's also the housekeeping task, which could generally be made available for such needs. It's basically idle after the init process and already has a large stack. I think about adding a callback execution queue to it, so my ticker handler would simply queue the notification generator call there.
Thanks, Michael
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
OvmsDev mailing list OvmsDev@lists.teslaclub.hk <mailto:OvmsDev@lists.teslaclub.hk> http://lists.teslaclub.hk/mailman/listinfo/ovmsdev <http://lists.teslaclub.hk/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
OvmsDev mailing list OvmsDev@lists.teslaclub.hk http://lists.teslaclub.hk/mailman/listinfo/ovmsdev
Am 28.03.2018 um 20:50 schrieb Michael Balzer:
My specific issue: I'm producing the notifications from my ticker1 handler. I now found out a simple standard info notification needs already around 10 ms, and my battery status update (15 data messages) needs 60-70 ms. I thought these would run faster, not sure where the time is spent. I'm using the command notifications, nice to use but with quite some overhead of course.
Tests and optimization results on this:
Replaced NotifyCommand() calls by NotifyString(), with strings produced by a reused BufferedShell and calling the output methods directly instead of doing the shell execution via microrl. That reduced the time to 40 ms.
Replaced the BufferedShells by StringWriters to also eliminate the memory overhead of the LogBuffer → std::string dump. I did that only for the memory, but it reduced the time to 20 ms.
So there is quite some overhead involved in the standard command execution now.
I recommend using string notifications for all time critical data updates (i.e. battery cell blocks).
Regards, Michael
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
participants (2)
-
Mark Webb-Johnson -
Michael Balzer