<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Firstly, remember that with memory debugging turned on (menuconfig, Components, OVMS, Developer, Enabled extended RAM memory allocation statistics), there is an extra overhead for each memory allocation. I tested a simple allocation of a 32 byte object, and that reduced memory by 56 bytes. So, seems to be 24 bytes for each allocation. Not sure what it is without the debugging turned on (as much harder to see the allocations), but it should be a lot less.</div><div class=""><br class=""></div><div class="">I added some instrumentation, and got this:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">I (2778) metrics: Initialising METRICS (1810)</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">I (2817) metrics: OvmsMetric is 28 bytes</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">I (2861) metrics: OvmsMetricBool is 32 bytes</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">I (2909) metrics: OvmsMetricInt is 32 bytes</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">I (2955) metrics: OvmsMetricFloat is 32 bytes</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">I (3003) metrics: OvmsMetricString is 52 bytes</span></font></div></div></blockquote><div class=""><br class=""></div><div class="">Then, a ‘test metric’ to register a new OvmsMetricBool metric, and got this:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">OVMS > test metric</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">OVMS > module memory</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">============================</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">Free 8-bit 102028/246616, 32-bit 18576/43548, numafter = 921</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">task=Asy total= 0 104 24972 change= +0 +104 +0</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">============================</span></font></div></div></blockquote><div class=""><br class=""></div><div class="">That is using a ‘x.test.metric’ name (which should fit in the std::string small string allocation).</div><div class=""><br class=""></div><div class="">It seems that using OvmsMetricBool (probably our smallest), we currently need 32bytes for the bool itself, plus 48 bytes for the OvmsMetrics map linkage, plus 24 bytes allocation overhead.</div><div class=""><br class=""></div><div class="">Looking at the OvmsMetricBool, it adds a ‘bool m_value’ on top of the base OvmsMetric values, which are:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> const char* m_name;</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> bool m_defined;</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> bool m_stale;</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> int m_autostale;</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> metric_unit_t m_units;</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> std::bitset<METRICS_MAX_MODIFIERS> m_modified;</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> uint32_t m_lastmodified;</span></font></div></div></blockquote><div class=""><br class=""></div><div class=""><div class="">Looking at the OvmsMetric member variables, we can live with a autostale of unsigned 16 bit integer, so re-arranging those is a simple win:</div><div class=""><br class=""></div><blockquote style="margin: 0px 0px 0px 40px; border: none; padding: 0px;" class=""><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> const char* m_name;</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> metric_unit_t m_units;</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> std::bitset<METRICS_MAX_MODIFIERS> m_modified;</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> uint32_t m_lastmodified;</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> uint16_t m_autostale;</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> bool m_defined;</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> bool m_stale;</span></font></div></blockquote><div class=""><br class=""></div><div class="">With that done, OvmsMetric goes from 28 bytes to 24 (with similar 4 byte win on all the other derived types). Also, my simple registration of a new OvmsMetricBool goes from 104 bytes to 100.</div></div><div class=""><br class=""></div><div class="">Looking at the m_metrics map storage, I tried a simple:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">std::forward_list<uint32_t> x;</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""><br class=""></span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">for (uint32_t k=0;k<100;k++)</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""> x.push_front(k);</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""><br class=""></span></font></div><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">OVMS > test metric</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">OVMS > module memory</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">task=Asy total= 0 3228 24972 change= +0 +3228 +0</span></font></div></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">OVMS > test metric</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">OVMS > module memory</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">task=Asy total= 0 6428 24972 change= +0 +3200 +0</span></font></div></blockquote><div class=""><br class=""></div><div class="">That works out at 32 bytes per std::forward_list entry. That is presumably the 24 bytes for the allocation, plus 8 bytes per entry (a pointer to the next, plus a pointer to the entry content).</div><div class=""><br class=""></div><div class="">The absolute most efficient dynamic system would be an OvmsMetric* m_next in OvmsMetric, then a OvmsMetric* m_first in OvmsMetrics, and remove the map altogether. That would be 4 bytes for the OvmsMetrics list, plus an extra 4 bytes for each metric.</div><div class=""><br class=""></div><div class=""><div class="">The original design made a lot of use of MyMetrics.Find(), but that has been deprecated now and I don’t see anything using it at all. The only thing we need is a ‘metrics list’ iterator to show the details - all that needs is an ordered list.</div></div><div class=""><br class=""></div><div class="">So, I went ahead and did that. Changed ovms_metrics to use a manually managed one-way linked list.</div><div class=""><br class=""></div><div class="">With the original code, and my default set of metrics (not including the Twizy ones), this comes to:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">OVMS > module memory</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">============================</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">Free 8-bit 102264/246616, 32-bit 18576/43548, numafter = 917</span></font></div></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""><br class=""></span></font></div><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">OVMS > vehicle module RT</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">I (107076) v-renaulttwizy: Renault Twizy vehicle module</span></font></div></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""><br class=""></span></font></div><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">OVMS > module memory</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">============================</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">Free 8-bit 75380/246616, 32-bit 18576/43548, numafter = 1000</span></font></div></div></blockquote><div class=""><br class=""></div><div class="">With the new code, and my default set of metrics (not including the Twizy ones), this comes to:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">OVMS > module memory</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">============================</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">Free 8-bit 106536/246632, 32-bit 18576/43548, numafter = 828</span></font></div></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""><br class=""></span></font></div><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">OVMS > vehicle module RT</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">I (55126) v-renaulttwizy: Renault Twizy vehicle module</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class=""><br class=""></span></font></div><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">OVMS > module memory</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">============================</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">Free 8-bit 85800/246632, 32-bit 18576/43548, numafter = 1000</span></font></div></div></div></blockquote><div class=""><br class=""></div><div class="">So, a reasonable saving of 10KB.</div><div class=""><br class=""></div><div class="">I tried with memory and task debugging turned off, and we get m.freeram 118888/98200 going to 121884/104932.</div><div class=""><br class=""></div><div class="">With the optimised OvmsMetrics code, RT seems to be allocating memory as follows:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">Free 8-bit 85812/246632, 32-bit 18576/43548, numafter = 1000</span></font></div><div class=""><font face="Andale Mono" class=""><span style="font-size: 18px;" class="">task=Asy total= 0 20616 24972 change= +0 +20616 +0</span></font></div></div></blockquote><div class=""><br class=""></div><div class="">We could use a static style allocation (std::vector, or a static array), and fixed structures (rather than dynamic objects). That would save the allocation overhead, but would make things a lot more rigid.</div><div class=""><br class=""></div><div class="">All the above committed and pushed.</div><div class=""><br class=""></div><div class="">Regards, Mark.</div><div class=""><br class=""></div><div class="">P.S. Note that Espressif are just now coming out with WROVER modules that include 32Mbit of PSRAM (can be mapped so that heap size goes up around 4MB!) Probably another few months before code support for that stabilises, and modules available in quantity. Perhaps in our future we could switch to that, but for the moment I think we’re ok. Just need to be careful.</div><div class=""><br class=""></div><div><blockquote type="cite" class=""><div class="">On 30 Oct 2017, at 5:45 AM, Michael Balzer <<a href="mailto:dexter@expeedo.de" class="">dexter@expeedo.de</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">While implementing the set type metrics I mentioned before, I've checked<br class="">out metrics RAM usage: just adding four new metrics reduced my free<br class="">memory by 656 bytes. 112 bytes of that were names, 16 bytes were actual<br class="">data content, 16 bytes were pointers to the metrics.<br class=""><br class="">So it seems each metric currently needs 128 bytes management overhead (+<br class="">RAM for the name in case of dynamic names like for an array of battery<br class="">cells).<br class=""><br class="">I've added 128 custom metrics now, which add up to ~ 20 KB of RAM --<br class="">more than 30% of the free RAM available before. I've got ~80% of my<br class="">planned metrics now, so it should fit for the Twizy, but a more complex<br class="">monitoring would need to use combined metrics for i.e. cell data.<br class=""><br class="">Maybe we can change the registry structure to not using std::map. The<br class="">fast btree lookup is not really needed if listeners use pointers. A<br class="">simple std::forward_list would do if new metrics are inserted sorted.<br class="">Also, allocating RAM for name patterns like "...cell.<n>..." is of<br class="">course very wasteful, maybe we can introduce some array logic to<br class="">generate these names.<br class=""><br class="">Regards,<br class="">Michael<br class=""><br class=""><br class="">Am 25.10.2017 um 07:09 schrieb Stephen Casner:<br class=""><blockquote type="cite" class="">I'd like to add another $.02 regarding RAM usage. We need to consider<br class="">the impact of all three modes of RAM usage:<br class=""><br class="">1. static declaration of variables<br class=""><br class="">2. local (automatic) variables that cause the maximum stack usage to<br class="">increase so we need to dedicate a larger stack allocation<br class=""><br class="">3. dynamic allocations from the heap<br class=""><br class="">I have more specific comments for each of these.<br class=""><br class="">1. If you need a large buffer, declaring it as static storage means<br class="">that it is always allocated even if your code is not being used<br class="">(unless it is something like vehicle-specific code that is configured<br class="">out). So, it would be better to dynamically allocate that buffer<br class="">space from the heap (malloc) when needed and then free it when<br class="">finished so that the usage is only temporary. That way the same space<br class="">might be used for other purposes at other times.<br class=""><br class="">2. I recomment NOT USING std::string except where it is really needed<br class="">and useful. In particular if you have a function parameter that is<br class="">always supplied as a character constant but the type of the parameter<br class="">is std::string then the compiler needs to expand the caller's stack<br class="">space by 32 bytes, for each such instance of the call, to hold the<br class="">std:string structure. Additional heap space is required for the<br class="">characters. None of that would be required if the parameter type were<br class="">const char*. The same problem applies to functions that return<br class="">std::string, since the compiler must allocate stack space in the<br class="">calling function for the return value to be copied. In particular if<br class="">the caller is just going to print that string with .c_str(), it would<br class="">be much better to put the .c_str() in the called function and return<br class="">the const char* UNDER ONE IMPORTANT CONDITION: this depends on the<br class="">std::string in the called function being stable, such as a member<br class="">variable of the class. If the string in the called function is<br class="">automatic (allocated on the stack), then the .c_str() of it won't be<br class="">valid in the caller.<br class=""><br class="">I saved substantial stack space and also heap space by changing the<br class="">command maps in OvmsCommand from map<std::string, OvmsCommand*> to<br class="">map<const char*, OvmsCommand*, CompareCharPtr>. This was possible<br class="">because all of the command token strings that are put in the map come<br class="">from character constants anyway, and those are stable.<br class=""><br class="">I think there are several more functions that could safely have their<br class="">arguments or return values changed. Now, I don't mean to be pushing<br class="">us back to essentially writing C code in C++ and ignoring the benefits<br class="">of C++. For places where dynamic storage is needed, as for a class<br class="">member, using std::string is a big advantage and not a problem. Just<br class="">be cognizant of the costs where it is used.<br class=""><br class="">3. As I mentioned in an earlier message, there is another 40K of RAM<br class="">available for dynamic allocation by code that only requires 32-bit<br class="">access, not byte-access. This is in IRAM (Instruction RAM). It won't<br class="">be allocated by 'malloc' or 'new' but can be allocated explicitly with<br class="">pvPortMallocCaps(sizeof(size, MALLOC_CAP_32BIT). I'm currently using<br class="">part of it for the storage of metadata about blocks allocated from the<br class="">heap in the ovms_module debugging code to minimize the impact that<br class="">using that code has on the memory available for the code to be tested.<br class=""><br class=""> -- Steve<br class="">_______________________________________________<br class="">OvmsDev mailing list<br class=""><a href="mailto:OvmsDev@lists.teslaclub.hk" class="">OvmsDev@lists.teslaclub.hk</a><br class="">http://lists.teslaclub.hk/mailman/listinfo/ovmsdev<br class=""></blockquote><br class="">-- <br class="">Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal<br class="">Fon 02333 / 833 5735 * Handy 0176 / 206 989 26<br class=""><br class=""><br class="">_______________________________________________<br class="">OvmsDev mailing list<br class=""><a href="mailto:OvmsDev@lists.teslaclub.hk" class="">OvmsDev@lists.teslaclub.hk</a><br class="">http://lists.teslaclub.hk/mailman/listinfo/ovmsdev<br class=""></div></div></blockquote></div><br class=""></body></html>