[Ovmsdev] 2019 / scripting framework
Mark Webb-Johnson
mark at webb-johnson.net
Wed Jan 16 15:22:23 HKT 2019
Well, the javascript framework re-work has now been committed, and is working well for me. Only one more major outstanding TODO remaining. Some comments:
To do anything useful with modules, I had to increase the stack size. It is now 12,288 bytes. I’ve seen just under 10KB in normal use, so a few more KB added as a margin. It won’t work with the old 8192 byte default. At least it doesn’t use IRAM, and the majority of storage (code, execution stack, etc) is in SPIRAM.
Duktape now starts up as part of the normal AutoInit system (so if it does crash, at least the rest of the system will continue to run).
Modules have been fully implemented. Very nice.
An internal module system has been implemented, and a first module 'int/pubsub' provided embedded in the firmware.
Breaking Change: The old ’script <path>’ command has changed to ’script run <path>’. The ‘.’ alternative is unchanged.
A ’script reload’ command has been provided to reload the framework.
A ’script eval’ command has been provided as a quick-and-dirty javascript evaluator (for testing purposes).
TODO: There is a memory leak in the PubSub event framework, so I’ve disabled it for the moment.
For example:
OVMS# script eval 'JSON=require("lib/JSON");'
OVMS# script eval 'JSON.print(this)'
{
"OvmsCommand": function () { [native code] },
"OvmsLocationStatus": function () { [native code] },
"OvmsMetricFloat": function () { [native code] },
"OvmsMetricValue": function () { [native code] },
"assert": function () { [native code] },
"print": function () { [native code] },
"PubSub": {
"publish": function () { [ecmascript code] },
"subscribe": function () { [ecmascript code] },
"clearAllSubscriptions": function () { [ecmascript code] },
"clearSubscriptions": function () { [ecmascript code] },
"unsubscribe": function () { [ecmascript code] }
},
"JSON": {
"print": function () { [ecmascript code] }
}
OVMS# script reload
Reloading javascript engine
I (277675) script: Duktape: Clearing existing context
I (277685) script: Duktape: Creating heap
I (277685) script: Duktape: Initialising module system
I (277685) script: Duktape: Pre-Registered function OvmsCommand
I (277685) script: Duktape: Pre-Registered function OvmsLocationStatus
I (277685) script: Duktape: Pre-Registered function OvmsMetricFloat
I (277685) script: Duktape: Pre-Registered function OvmsMetricValue
I (277685) script: Duktape: Pre-Registered function assert
I (277685) script: Duktape: Pre-Registered function print
I (277685) script: Duktape: Preload internal module PubSub
I (277825) script: Duktape: Executing ovmsmain.js
I (277835) script: Hello world
Still a bunch to do, but the basics seem there and working well.
Regards, Mark.
> On 11 Jan 2019, at 9:23 AM, Mark Webb-Johnson <mark at webb-johnson.net> wrote:
>
> Your example is a little more involved than my ‘hello world’, and much cooler. Perhaps Robert can use it as a starting point in the user guide.
>
> Any ideas how to handle events? We need an event module in javascript, and some way for extension scripts/modules to hook into the reception of events? The events are already routed into the duktape task - just not decided what to do with them yet. This leads into the bigger question of a server-side framework to run in the Duktape task. I can see some common pub/sub frameworks:
>
> PubSub.JS <https://github.com/mroderick/PubSubJS>
> Subtopic <https://github.com/pmelander/Subtopic>
> Amplify JS <http://amplifyjs.com/api/pubsub/>
> Postal JS <https://github.com/postaljs/postal.js>
>
> The last one (postal js) seems a good balance of complexity vs functionality, but goes beyond the minimum we need and may be too big. The first one is the absolute minimum, and the compressed (min) version is <2KB.
>
> I guess the bigger question is does anyone know of any very lightweight server side frameworks that we should be looking at (rather than rolling our own - aka OpenVehicles.JS)?
>
> It also would seem to make more sense to have the C interfaces as module objects with methods. So, rather than calling OvmsMetricFloat(), you get a handle to a metric object and then call a method to obtain its float value. Not sure how Duktape handles that, but I will check. Having a rich set of interfaces to our objects would be very useful and clean. Another example would be the vehicle object with methods StartCharge, StopCharge, UnlockCar, etc - rather than calling OvmsCommand(“charge start").
>
> Regards, Mark.
>
>> On 11 Jan 2019, at 12:38 AM, Michael Balzer <dexter at expeedo.de <mailto:dexter at expeedo.de>> wrote:
>>
>> Works like a charm :)
>>
>> I've added the autoinit option to the web UI.
>>
>> Important for all script writers is to keep in mind we now have a global shared context.
>>
>> That is very useful to share data between scripts, but needs a little bit of discipline to avoid polluting the global context with local variables and functions. Any "var", "const", "function" etc. placed into the top level of a script file now is added to the global context.
>>
>> The best approach is to encourage users to use closures by default for all their scripts, i.e.
>> (function(){
>> … user code …
>> })();
>> The same rule applies to web plugins.
>>
>> Node style modules also work nicely. A simple test module:
>> // Module : JSON (not compatible with the browser component!)
>> // State : test/demo
>> // Install as : /store/scripts/lib/JSON.js
>> // Load : JSON = require("lib/JSON");
>> // Use : JSON.print(object);
>>
>> exports.print = function(obj, ind) {
>> var type = typeof obj;
>> if (type == "object" && Array.isArray(obj)) type = "array";
>> if (!ind) ind = '';
>>
>> switch (type) {
>> case "string":
>> print('"' + obj.replace(/\"/g, '\\\"') + '"');
>> break;
>> case "array":
>> print('[\n');
>> for (var i = 0; i < obj.length; i++) {
>> print(ind + ' ');
>> exports.print(obj[i], ind + ' ');
>> if (i != obj.length-1) print(',');
>> print('\n');
>> }
>> print(ind + ']');
>> break;
>> case "object":
>> print('{\n');
>> var keys = Object.keys(obj);
>> for (var i = 0; i < keys.length; i++) {
>> print(ind + ' "' + keys[i] + '": ');
>> exports.print(obj[keys[i]], ind + ' ');
>> if (i != keys.length-1) print(',');
>> print('\n');
>> }
>> print(ind + '}');
>> break;
>> default:
>> print(obj);
>> }
>>
>> if (ind == '') print('\n');
>> }
>>
>> Using it:
>> JSON = require("lib/JSON");
>>
>> print("Global context:\n");
>> JSON.print(this);
>>
>> (function(){
>> var x = { a: 42, b: "a \"foo\" is no 'bar'", c: [1,2,3], d: { sub: true }, e: ["q","w",17, { e1:22, e2:33 }] };
>> print("\nLocal object:\n");
>> JSON.print(x);
>> })();
>>
>> Execution:
>> OVMS# . test.js
>> Global context:
>> {
>> "print": function () { [native code] },
>> "assert": function () { [native code] },
>> "OvmsMetricValue": function () { [native code] },
>> "OvmsMetricFloat": function () { [native code] },
>> "OvmsLocationStatus": function () { [native code] },
>> "OvmsCommand": function () { [native code] },
>> "JSON": {
>> "print": function () { [ecmascript code] }
>> }
>> }
>>
>> Local object:
>> {
>> "a": 42,
>> "b": "a \"foo\" is no 'bar'",
>> "c": [
>> 1,
>> 2,
>> 3
>> ],
>> "d": {
>> "sub": true
>> },
>> "e": [
>> "q",
>> "w",
>> 17,
>> {
>> "e1": 22,
>> "e2": 33
>> }
>> ]
>> }
>>
>> Nice :)
>>
>> Regards,
>> Michael
>>
>>
>> Am 10.01.19 um 10:10 schrieb Mark Webb-Johnson:
>>> I’ve committed my work on a production javascript framework. The changes are quite extensive, and with the one exception of ‘ovmsprint’, should be backwards compatible.
>>>
>>> The overall goal is to be able to run a javascript code system alongside our standard firmware code. Users could then write javascript, and modify scripts already running on the device (without firmware changes). Unlike the previous version (where scripts were loaded, compiled, run, then dumped), this approach is designed to keep scripts in memory; this vastly increases the opportunities for what can be done with the system.
>>>
>>> Changes made include:
>>>
>>> Move scripting to be a component
>>> Make DukTape use SPIRAM for as much as possible
>>> Run the javascript engine in it's own task (OVMS DukTape)
>>> Change the way extensions functions are registered
>>> (just call RegisterDuktapeFunction, rather than messing around with the internals of Duktape)
>>> Catch compilation and parsing errors (fail gracefully - finally!)
>>> Support 'print' and 'assert' javascript framework
>>> Output (via print) goes to current console, or logged if no console
>>> Autoinit (and run) /store/scripts/ovmsmain.js
>>> Support node.js style modules
>>> Control with config auto javascript (default enable)
>>>
>>> The main TODO is to provide a facility for reloading the engine (new scripts/modules). At the moment, the module needs to be rebooted (ugly). There is also a lot of work still to do on making some actually useful modules. And we also need to work out what to do with events (now that they can be delivered to running javascripts).
>>>
>>> But, anyway, for the moment:
>>>
>>> I (128) script: Initialising SCRIPTS (1600)
>>> I (132) script: Using DUKTAPE javascript engine
>>> I (0) script: Duktape: Creating heap
>>> I (10) script: Duktape: Initialising module system
>>> I (20) script: Duktape: Scripting task is running
>>> I (906) script: Duktape: Executing ovmsmain.js
>>> I (936) script: Hello world
>>>
>>> OVMS# vfs cat /store/scripts/ovmsmain.js
>>> print("Hello world\n”);
>>>
>>> OVMS# test javascript
>>> Javascript 1+2=3
>>>
>>> Regards, Mark.
>>>
>>>> On 8 Jan 2019, at 11:00 PM, Michael Balzer <dexter at expeedo.de <mailto:dexter at expeedo.de>> wrote:
>>>>
>>>> Mark,
>>>>
>>>> welcome back, awesome news on the JS and FCC stuff :)
>>>>
>>>> I've got some fixes and some feedback on the web plugins in progress, I'll get that done until the weekend (got some free days left).
>>>>
>>>> Regards,
>>>> Michael
>>>>
>>>>
>>>> Am 08.01.19 um 05:27 schrieb Mark Webb-Johnson:
>>>>> Firstly, a very belated merry xmas, and happy 2019 to everyone. I hope that you all got to spend good times with friends and family, as I did. I’ve been travelling, and just got back home to a very large eMail inbox.
>>>>>
>>>>> During my travels, I did manage to re-work the javascript framework to be single task based. That seems to work well, and now I’m back I will test on my car. The new framework is backwards compatible - apart from memory usage (one more task with a stack), it has no impact to the flow. It will, however, allow us to extend this a lot more and become much more useful. Assuming no issues, I will be able to commit that soon.
>>>>>
>>>>> As we are up to 144 commits since 3.1.011, perhaps it is well past due for 3.1.012? I was hoping that Espressif would have fixed the bug related to wear levelling version upgrades that Michael helped to identify, but not yet. Any objections to a 3.1.012 release to EAP at the end of this week?
>>>>>
>>>>> The CE/FCC certification work is progressing. No issues so far. I’ll let you know as soon as we get near completion for this.
>>>>>
>>>>> I’m working through my inbox now, and will reply individually to questions there.
>>>>>
>>>>> Regards, Mark.
>>>>>
>>>>> _______________________________________________
>>>>> OvmsDev mailing list
>>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>>>
>>>> --
>>>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>>>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>>>>
>>>>
>>>> _______________________________________________
>>>> OvmsDev mailing list
>>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>>
>>>
>>>
>>> _______________________________________________
>>> OvmsDev mailing list
>>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
>>
>> --
>> Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
>> Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
>> _______________________________________________
>> OvmsDev mailing list
>> OvmsDev at lists.openvehicles.com <mailto:OvmsDev at lists.openvehicles.com>
>> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
>
> _______________________________________________
> OvmsDev mailing list
> OvmsDev at lists.openvehicles.com
> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvehicles.com/pipermail/ovmsdev/attachments/20190116/dfa6b1ef/attachment.htm>
More information about the OvmsDev
mailing list