I feel that our plugin system offers a missed opportunity. By making small extensions to available functionality, we can use this system to do so much - and with a much lower complexity and barrier to entry compared to C++ ESP IDF development. There are a couple of things I would like to now do: Implement a mechanism for plugins to directly extend the console command language (other than the current ’script exec PLUGIN.fn’). Implement a plugin repository, and command line tools for managing plugins. We simply need to have a store and directory of available plugins, a way of searching/listing those, and a uniform way to install/remove plugins. Once this is done, perhaps someone can handle the web UI side of this, and perhaps that can even be done as a plugin :-) I’m already working on #1. Just an extension to the existing script system to allow plugins to register commands, and to have the command register/deregister done automatically on plugin load/unload. However, #2 is a little more tricky and I’m looking for feedback on the design of this. My idea is: Store the plugins in a new github/openvehicles/Open-Vehicle-Monitoring-System-3/plugins folder. Eventually the /plugin folder will all be migrated to here. Each plugin has a unique name; the directory in /plugins. Within each plugin directory, there is a <name>.json file (JSON format) with defined tags to describe the plugin. If the plugin delivers a web extension, that is in the ‘web’ subdirectory. If the plugin delivered a script extension, that is in the ’script’ directory. Documentation (in rst format) is in the ‘docs' directory. The JSON file would need the following, at a minimum: Name: The plugin ID Title: One line title Group: Group for the plugin Version: The version string of the plugin Description: Longer textual description of the plugin Prerequisites: Prerequisites to be checked (Each plugin installed provides name=version, the system provides system-level ones such as the version of firmware, and then the prerequisites is simple a list of <name><operator><version> as required) We can automatically build the documentation, and list, for all plugins for our docs.openvehicles.com <http://docs.openvehicles.com/> (in a similar way to how it is done for /plugin now). On a repository server, as part of a nightly build job, we can automatically build a single directory JSON file from the <name>,json descriptors. We can publish this, as well as a git export/checkout of the plugins, on a http download site. On the module, we can store a list of repository servers (allowing the user to add/remove as they require). Contributors/authors of plugins simply GitHub clone our repository, and issue pull requests as appropriate. They can also have their own private repository servers (just need to host the directory json file and associated code files) and add it to their modules. The ‘plugin’ command tree on the module will provide commands for: List installed repositories Add a new repository Remove a repository List installed plugins List/Search available plugins Install a plugin Remove a plugin Update a plugin I would like to try to unify the web and script plugin systems a bit more closely. They are really two sides of the same coin (with some plugins providing both). Does that make sense? Any suggestions/comments? Regards, Mark.
This sounds great! Would really ease up the installation process and allow less experienced users to add the additional features. Regards, Jaunius Kapkan [Sent from cellphone]
On 2020-07-14, at 08:57, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
I feel that our plugin system offers a missed opportunity. By making small extensions to available functionality, we can use this system to do so much - and with a much lower complexity and barrier to entry compared to C++ ESP IDF development.
There are a couple of things I would like to now do:
Implement a mechanism for plugins to directly extend the console command language (other than the current ’script exec PLUGIN.fn’).
Implement a plugin repository, and command line tools for managing plugins. We simply need to have a store and directory of available plugins, a way of searching/listing those, and a uniform way to install/remove plugins. Once this is done, perhaps someone can handle the web UI side of this, and perhaps that can even be done as a plugin :-)
I’m already working on #1. Just an extension to the existing script system to allow plugins to register commands, and to have the command register/deregister done automatically on plugin load/unload.
However, #2 is a little more tricky and I’m looking for feedback on the design of this. My idea is:
Store the plugins in a new github/openvehicles/Open-Vehicle-Monitoring-System-3/plugins folder. Eventually the /plugin folder will all be migrated to here.
Each plugin has a unique name; the directory in /plugins.
Within each plugin directory, there is a <name>.json file (JSON format) with defined tags to describe the plugin. If the plugin delivers a web extension, that is in the ‘web’ subdirectory. If the plugin delivered a script extension, that is in the ’script’ directory. Documentation (in rst format) is in the ‘docs' directory. The JSON file would need the following, at a minimum:
Name: The plugin ID Title: One line title Group: Group for the plugin Version: The version string of the plugin Description: Longer textual description of the plugin Prerequisites: Prerequisites to be checked (Each plugin installed provides name=version, the system provides system-level ones such as the version of firmware, and then the prerequisites is simple a list of <name><operator><version> as required)
We can automatically build the documentation, and list, for all plugins for our docs.openvehicles.com (in a similar way to how it is done for /plugin now).
On a repository server, as part of a nightly build job, we can automatically build a single directory JSON file from the <name>,json descriptors. We can publish this, as well as a git export/checkout of the plugins, on a http download site.
On the module, we can store a list of repository servers (allowing the user to add/remove as they require).
Contributors/authors of plugins simply GitHub clone our repository, and issue pull requests as appropriate. They can also have their own private repository servers (just need to host the directory json file and associated code files) and add it to their modules.
The ‘plugin’ command tree on the module will provide commands for:
List installed repositories Add a new repository Remove a repository List installed plugins List/Search available plugins Install a plugin Remove a plugin Update a plugin
I would like to try to unify the web and script plugin systems a bit more closely. They are really two sides of the same coin (with some plugins providing both).
Does that make sense? Any suggestions/comments?
Regards, Mark.
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Great idea!The only thing I would probably add is "tags" in the json file for documenting the plugins.Nikolay On Tuesday, July 14, 2020, 10:09:28 AM GMT+2, Jaunius Kapkan <jaunius@gmx.com> wrote: This sounds great! Would really ease up the installation process and allow less experienced users to add the additional features. Regards, Jaunius Kapkan [Sent from cellphone] On 2020-07-14, at 08:57, Mark Webb-Johnson <mark@webb-johnson.net> wrote: I feel that our plugin system offers a missed opportunity. By making small extensions to available functionality, we can use this system to do so much - and with a much lower complexity and barrier to entry compared to C++ ESP IDF development. There are a couple of things I would like to now do: - Implement a mechanism for plugins to directly extend the console command language (other than the current ’script exec PLUGIN.fn’). - Implement a plugin repository, and command line tools for managing plugins. We simply need to have a store and directory of available plugins, a way of searching/listing those, and a uniform way to install/remove plugins. Once this is done, perhaps someone can handle the web UI side of this, and perhaps that can even be done as a plugin :-) I’m already working on #1. Just an extension to the existing script system to allow plugins to register commands, and to have the command register/deregister done automatically on plugin load/unload. However, #2 is a little more tricky and I’m looking for feedback on the design of this. My idea is: - Store the plugins in a new github/openvehicles/Open-Vehicle-Monitoring-System-3/plugins folder. Eventually the /plugin folder will all be migrated to here. - Each plugin has a unique name; the directory in /plugins. - Within each plugin directory, there is a <name>.json file (JSON format) with defined tags to describe the plugin. If the plugin delivers a web extension, that is in the ‘web’ subdirectory. If the plugin delivered a script extension, that is in the ’script’ directory. Documentation (in rst format) is in the ‘docs' directory. The JSON file would need the following, at a minimum: - Name: The plugin ID - Title: One line title - Group: Group for the plugin - Version: The version string of the plugin - Description: Longer textual description of the plugin - Prerequisites: Prerequisites to be checked (Each plugin installed provides name=version, the system provides system-level ones such as the version of firmware, and then the prerequisites is simple a list of <name><operator><version> as required) - We can automatically build the documentation, and list, for all plugins for our docs.openvehicles.com (in a similar way to how it is done for /plugin now). - On a repository server, as part of a nightly build job, we can automatically build a single directory JSON file from the <name>,json descriptors. We can publish this, as well as a git export/checkout of the plugins, on a http download site. - On the module, we can store a list of repository servers (allowing the user to add/remove as they require). - Contributors/authors of plugins simply GitHub clone our repository, and issue pull requests as appropriate. They can also have their own private repository servers (just need to host the directory json file and associated code files) and add it to their modules. - The ‘plugin’ command tree on the module will provide commands for: - List installed repositories - Add a new repository - Remove a repository - List installed plugins - List/Search available plugins - Install a plugin - Remove a plugin - Update a plugin - I would like to try to unify the web and script plugin systems a bit more closely. They are really two sides of the same coin (with some plugins providing both). Does that make sense? Any suggestions/comments? Regards, Mark. _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Mark, full ack, has been on my list as well. I'll handle the web UI side. I don't think we need to migrate/change the folder structure, we should be able to just add the meta data. I tried to define the structure to be usable as a public repository. Did I miss something? Btw, I used the naming "module" plugins instead of "script" plugins, as web plugins also consist mostly of scripts. I thought that would clear things for users, but I'm not sure now. Regarding the plugin structure and meta data, we should decide wether to split plugins into their elements or keep them bundled. You seem to opt for bundles ("web subdirectory"). I would second that, but we then need to take into account a plugin may consist of multiple module and/or web plugins (see Edimax plugin for an example). Normally it's just one module plugin, but we should not restrict the system in that way. So the JSON file should allow to define an array of plugin elements / options. Currently there's a version on each element, and but we could/should simplify this to a central overall version for the plugin. To automatically install plugins, the meta data also needs to define the types and default setups for web/frontend plugins. Draft: edimax.json: { name: "edimax", title: "Edimax: Smart Plug Control", description: "Smart Plug control for Edimax models SP-1101W, SP-2101W et al", group: "Home Automation", info: "https://docs.openvehicles.com/en/latest/plugin/edimax/README.html", maintainer: "Michael Balzer <dexter@dexters-web.de>", version: "2.1", prerequisites: ["ovms>=3.2.010"], elements: [{ type: "script", path: "edimax.js", name: "edimax" // optional, defaults to plugin name },{ type: "webpage", path: "edimax.htm", page: "/usr/edimax", label: "Edimax Smart Plug", menu: "Config", auth: "Cookie" },{ type: "webhook", path: "edimax-status.htm", page: "/status", hook: "body.post" }] } A method to easily disable/enable installed plugins would also be nice, i.e. as subcommands "plugin disable" / "plugin enable". Regards, Michael Am 14.07.20 um 07:55 schrieb Mark Webb-Johnson:
I feel that our plugin system offers a missed opportunity. By making small extensions to available functionality, we can use this system to do so much - and with a much lower complexity and barrier to entry compared to C++ ESP IDF development.
There are a couple of things I would like to now do:
1. Implement a mechanism for plugins to directly extend the console command language (other than the current ’script exec PLUGIN.fn’).
2. Implement a plugin repository, and command line tools for managing plugins. We simply need to have a store and directory of available plugins, a way of searching/listing those, and a uniform way to install/remove plugins. Once this is done, perhaps someone can handle the web UI side of this, and perhaps that can even be done as a plugin :-)
I’m already working on #1. Just an extension to the existing script system to allow plugins to register commands, and to have the command register/deregister done automatically on plugin load/unload.
However, #2 is a little more tricky and I’m looking for feedback on the design of this. My idea is:
* Store the plugins in a new github/openvehicles/Open-Vehicle-Monitoring-System-3/plugins folder. Eventually the /plugin folder will all be migrated to here.
* Each plugin has a unique name; the directory in /plugins.
* Within each plugin directory, there is a <name>.json file (JSON format) with defined tags to describe the plugin. If the plugin delivers a web extension, that is in the ‘web’ subdirectory. If the plugin delivered a script extension, that is in the ’script’ directory. Documentation (in rst format) is in the ‘docs' directory. The JSON file would need the following, at a minimum:
o Name: The plugin ID o Title: One line title o Group: Group for the plugin o Version: The version string of the plugin o Description: Longer textual description of the plugin o Prerequisites: Prerequisites to be checked (Each plugin installed provides name=version, the system provides system-level ones such as the version of firmware, and then the prerequisites is simple a list of <name><operator><version> as required)
* We can automatically build the documentation, and list, for all plugins for our docs.openvehicles.com <http://docs.openvehicles.com> (in a similar way to how it is done for /plugin now).
* On a repository server, as part of a nightly build job, we can automatically build a single directory JSON file from the <name>,json descriptors. We can publish this, as well as a git export/checkout of the plugins, on a http download site.
* On the module, we can store a list of repository servers (allowing the user to add/remove as they require).
* Contributors/authors of plugins simply GitHub clone our repository, and issue pull requests as appropriate. They can also have their own private repository servers (just need to host the directory json file and associated code files) and add it to their modules.
* The ‘plugin’ command tree on the module will provide commands for:
o List installed repositories o Add a new repository o Remove a repository o List installed plugins o List/Search available plugins o Install a plugin o Remove a plugin o Update a plugin
* I would like to try to unify the web and script plugin systems a bit more closely. They are really two sides of the same coin (with some plugins providing both).
Does that make sense? Any suggestions/comments?
Regards, Mark.
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26
Thanks, all, for the feedback. @Nikolay: tags make sense and we can add. I think that in general the json could contain other fields (comments, notes, anything). The system itself really just needs name+version, with everything else as decoration or an enhancement to help searching. @Derek: I’ll see how it looks when I upload. For download, a single zip is certainly easier, and this is not hard to do as part of the server script to publish a repository. Just not sure if it is actually necessary for us. Thanks for the link to the kodi repository structure. @Michael: I think ‘module’ is the correct word (rather than ’script'. Let’s stick with that. I’ll work on this module side first, as I am not so familiar with the web extensions. The reason I suggested the ‘plugins’ directory (rather than ‘plugin’) was to allow us to move to this on an individual plugin basis. I’m not sure if that is necessary (we may instead be able to work off the existence of the json master file). I plan to try to make one plugin json and format first, then sort out the server side repository structure for that. Then we can review. Regards, Mark.
On 14 Jul 2020, at 7:45 PM, Michael Balzer <dexter@expeedo.de> wrote:
Mark,
full ack, has been on my list as well. I'll handle the web UI side.
I don't think we need to migrate/change the folder structure, we should be able to just add the meta data. I tried to define the structure to be usable as a public repository. Did I miss something?
Btw, I used the naming "module" plugins instead of "script" plugins, as web plugins also consist mostly of scripts. I thought that would clear things for users, but I'm not sure now.
Regarding the plugin structure and meta data, we should decide wether to split plugins into their elements or keep them bundled. You seem to opt for bundles ("web subdirectory"). I would second that, but we then need to take into account a plugin may consist of multiple module and/or web plugins (see Edimax plugin for an example). Normally it's just one module plugin, but we should not restrict the system in that way. So the JSON file should allow to define an array of plugin elements / options.
Currently there's a version on each element, and but we could/should simplify this to a central overall version for the plugin.
To automatically install plugins, the meta data also needs to define the types and default setups for web/frontend plugins.
Draft: edimax.json:
{ name: "edimax", title: "Edimax: Smart Plug Control", description: "Smart Plug control for Edimax models SP-1101W, SP-2101W et al", group: "Home Automation", info: "https://docs.openvehicles.com/en/latest/plugin/edimax/README.html" <https://docs.openvehicles.com/en/latest/plugin/edimax/README.html>, maintainer: "Michael Balzer <dexter@dexters-web.de> <mailto:dexter@dexters-web.de>", version: "2.1", prerequisites: ["ovms>=3.2.010"], elements: [{ type: "script", path: "edimax.js", name: "edimax" // optional, defaults to plugin name },{ type: "webpage", path: "edimax.htm", page: "/usr/edimax", label: "Edimax Smart Plug", menu: "Config", auth: "Cookie" },{ type: "webhook", path: "edimax-status.htm", page: "/status", hook: "body.post" }] }
A method to easily disable/enable installed plugins would also be nice, i.e. as subcommands "plugin disable" / "plugin enable".
Regards, Michael
Am 14.07.20 um 07:55 schrieb Mark Webb-Johnson:
I feel that our plugin system offers a missed opportunity. By making small extensions to available functionality, we can use this system to do so much - and with a much lower complexity and barrier to entry compared to C++ ESP IDF development.
There are a couple of things I would like to now do:
Implement a mechanism for plugins to directly extend the console command language (other than the current ’script exec PLUGIN.fn’).
Implement a plugin repository, and command line tools for managing plugins. We simply need to have a store and directory of available plugins, a way of searching/listing those, and a uniform way to install/remove plugins. Once this is done, perhaps someone can handle the web UI side of this, and perhaps that can even be done as a plugin :-)
I’m already working on #1. Just an extension to the existing script system to allow plugins to register commands, and to have the command register/deregister done automatically on plugin load/unload.
However, #2 is a little more tricky and I’m looking for feedback on the design of this. My idea is:
Store the plugins in a new github/openvehicles/Open-Vehicle-Monitoring-System-3/plugins folder. Eventually the /plugin folder will all be migrated to here.
Each plugin has a unique name; the directory in /plugins.
Within each plugin directory, there is a <name>.json file (JSON format) with defined tags to describe the plugin. If the plugin delivers a web extension, that is in the ‘web’ subdirectory. If the plugin delivered a script extension, that is in the ’script’ directory. Documentation (in rst format) is in the ‘docs' directory. The JSON file would need the following, at a minimum:
Name: The plugin ID Title: One line title Group: Group for the plugin Version: The version string of the plugin Description: Longer textual description of the plugin Prerequisites: Prerequisites to be checked (Each plugin installed provides name=version, the system provides system-level ones such as the version of firmware, and then the prerequisites is simple a list of <name><operator><version> as required)
We can automatically build the documentation, and list, for all plugins for our docs.openvehicles.com <http://docs.openvehicles.com/> (in a similar way to how it is done for /plugin now).
On a repository server, as part of a nightly build job, we can automatically build a single directory JSON file from the <name>,json descriptors. We can publish this, as well as a git export/checkout of the plugins, on a http download site.
On the module, we can store a list of repository servers (allowing the user to add/remove as they require).
Contributors/authors of plugins simply GitHub clone our repository, and issue pull requests as appropriate. They can also have their own private repository servers (just need to host the directory json file and associated code files) and add it to their modules.
The ‘plugin’ command tree on the module will provide commands for:
List installed repositories Add a new repository Remove a repository List installed plugins List/Search available plugins Install a plugin Remove a plugin Update a plugin
I would like to try to unify the web and script plugin systems a bit more closely. They are really two sides of the same coin (with some plugins providing both).
Does that make sense? Any suggestions/comments?
Regards, Mark.
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26 _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
The for-v3.3 branch now has a simple working plugin repository. While it can download assets of all types (json, module, web_page, web_hook, and web_rsc), it currently only supports installation and enable/disable for scripts. Some notes: Repositories are used to store plugins. The repository has a name and a url, and is accessed over http/https for download. A repository ‘openvehicles’ at 'http://api.openvehicles.com/plugins’ is installed by default. That repository published the OVMS v3 firmware plugins from our GitHub repository. Repositories need to publish ‘plugins.json’ and ‘plugins.rev’ files in the root of their hosted directory. The first is JSON array of all the plugins hosted, and the second is a version number string for the repository. I’ve added a sample script to build the .json and .rev files from GitHub, and publish appropriately, in vehicle/OVMS.V3/support/buildplugins.pl. You can add custom repositories with ‘plugin repo install’, and remove with ‘plugin repo remove’. The repository meta data is cached in RAM whenever a repository command is run (or you can force a refresh with ‘plugin repo refresh’. OVMS# plugin repo list Repository Version openvehicles 3.2.014-98-g17b7989 Available plugins can be shown with ‘plugin list’. OVMS# plugin list Plugin Repo Version Description chgthrottle openvehicles 1.2 Throttle charge current and/or stop charge if charger gets too hot. helloworld openvehicles 0.1 This plugin is provided for demonstration and example purposes only. You can use ‘plugin install <name>’ to install a plugin, and ‘plugin remove <name>’ to remove it. The system will automatically download the plugin from the repository for you. Plugins can be enabled/disabled with ‘plugin enable <name>’ and ‘plugin disable <name>’, and updated with ‘plugin update {<name>}’ (if <name> is ommitted, all plugins are updated). Installing and enabling a plugin that provides javascript modules will now result in those modules automatically being loaded at duktape initialisation time. You don’t need to add them to ovmsmain.js. Both wifi and cellular network connections are supported. There is a ‘helloworld’ plugin now available as an example. OVMS# plugin install helloworld Plugin: helloworld installed ok I (620571) pluginstore: Downloading plugin: helloworld, with 2 element(s) I (621671) pluginstore: Element: helloworld.js downloaded ok I (622441) pluginstore: Element: helloworld.json downloaded ok I (622441) pluginstore: Plugin: helloworld downloaded ok OVMS# script reload Reloading javascript engine I (635121) ovms-duktape: Duktape: Clearing existing context I (635251) ovms-duktape: Duktape: Creating heap I (635351) ovms-duktape: Duktape: Initialising module system I (635601) pluginstore: Loading enabled plugins I (635601) pluginstore: Load helloworld (0.1) OVMS# script eval 'helloworld.info()' Hello world Next on my list is adding support for installing, enabling, and disabling web plugins; but I need to understand better how they integrate before I handle that. After that, I need to handle the dependency system, and this should be done. Regards, Mark.
On 16 Jul 2020, at 8:52 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Thanks, all, for the feedback.
@Nikolay: tags make sense and we can add. I think that in general the json could contain other fields (comments, notes, anything). The system itself really just needs name+version, with everything else as decoration or an enhancement to help searching.
@Derek: I’ll see how it looks when I upload. For download, a single zip is certainly easier, and this is not hard to do as part of the server script to publish a repository. Just not sure if it is actually necessary for us. Thanks for the link to the kodi repository structure.
@Michael: I think ‘module’ is the correct word (rather than ’script'. Let’s stick with that. I’ll work on this module side first, as I am not so familiar with the web extensions. The reason I suggested the ‘plugins’ directory (rather than ‘plugin’) was to allow us to move to this on an individual plugin basis. I’m not sure if that is necessary (we may instead be able to work off the existence of the json master file). I plan to try to make one plugin json and format first, then sort out the server side repository structure for that. Then we can review.
Regards, Mark.
On 14 Jul 2020, at 7:45 PM, Michael Balzer <dexter@expeedo.de <mailto:dexter@expeedo.de>> wrote:
Mark,
full ack, has been on my list as well. I'll handle the web UI side.
I don't think we need to migrate/change the folder structure, we should be able to just add the meta data. I tried to define the structure to be usable as a public repository. Did I miss something?
Btw, I used the naming "module" plugins instead of "script" plugins, as web plugins also consist mostly of scripts. I thought that would clear things for users, but I'm not sure now.
Regarding the plugin structure and meta data, we should decide wether to split plugins into their elements or keep them bundled. You seem to opt for bundles ("web subdirectory"). I would second that, but we then need to take into account a plugin may consist of multiple module and/or web plugins (see Edimax plugin for an example). Normally it's just one module plugin, but we should not restrict the system in that way. So the JSON file should allow to define an array of plugin elements / options.
Currently there's a version on each element, and but we could/should simplify this to a central overall version for the plugin.
To automatically install plugins, the meta data also needs to define the types and default setups for web/frontend plugins.
Draft: edimax.json:
{ name: "edimax", title: "Edimax: Smart Plug Control", description: "Smart Plug control for Edimax models SP-1101W, SP-2101W et al", group: "Home Automation", info: "https://docs.openvehicles.com/en/latest/plugin/edimax/README.html" <https://docs.openvehicles.com/en/latest/plugin/edimax/README.html>, maintainer: "Michael Balzer <dexter@dexters-web.de> <mailto:dexter@dexters-web.de>", version: "2.1", prerequisites: ["ovms>=3.2.010"], elements: [{ type: "script", path: "edimax.js", name: "edimax" // optional, defaults to plugin name },{ type: "webpage", path: "edimax.htm", page: "/usr/edimax", label: "Edimax Smart Plug", menu: "Config", auth: "Cookie" },{ type: "webhook", path: "edimax-status.htm", page: "/status", hook: "body.post" }] }
A method to easily disable/enable installed plugins would also be nice, i.e. as subcommands "plugin disable" / "plugin enable".
Regards, Michael
Am 14.07.20 um 07:55 schrieb Mark Webb-Johnson:
I feel that our plugin system offers a missed opportunity. By making small extensions to available functionality, we can use this system to do so much - and with a much lower complexity and barrier to entry compared to C++ ESP IDF development.
There are a couple of things I would like to now do:
Implement a mechanism for plugins to directly extend the console command language (other than the current ’script exec PLUGIN.fn’).
Implement a plugin repository, and command line tools for managing plugins. We simply need to have a store and directory of available plugins, a way of searching/listing those, and a uniform way to install/remove plugins. Once this is done, perhaps someone can handle the web UI side of this, and perhaps that can even be done as a plugin :-)
I’m already working on #1. Just an extension to the existing script system to allow plugins to register commands, and to have the command register/deregister done automatically on plugin load/unload.
However, #2 is a little more tricky and I’m looking for feedback on the design of this. My idea is:
Store the plugins in a new github/openvehicles/Open-Vehicle-Monitoring-System-3/plugins folder. Eventually the /plugin folder will all be migrated to here.
Each plugin has a unique name; the directory in /plugins.
Within each plugin directory, there is a <name>.json file (JSON format) with defined tags to describe the plugin. If the plugin delivers a web extension, that is in the ‘web’ subdirectory. If the plugin delivered a script extension, that is in the ’script’ directory. Documentation (in rst format) is in the ‘docs' directory. The JSON file would need the following, at a minimum:
Name: The plugin ID Title: One line title Group: Group for the plugin Version: The version string of the plugin Description: Longer textual description of the plugin Prerequisites: Prerequisites to be checked (Each plugin installed provides name=version, the system provides system-level ones such as the version of firmware, and then the prerequisites is simple a list of <name><operator><version> as required)
We can automatically build the documentation, and list, for all plugins for our docs.openvehicles.com <http://docs.openvehicles.com/> (in a similar way to how it is done for /plugin now).
On a repository server, as part of a nightly build job, we can automatically build a single directory JSON file from the <name>,json descriptors. We can publish this, as well as a git export/checkout of the plugins, on a http download site.
On the module, we can store a list of repository servers (allowing the user to add/remove as they require).
Contributors/authors of plugins simply GitHub clone our repository, and issue pull requests as appropriate. They can also have their own private repository servers (just need to host the directory json file and associated code files) and add it to their modules.
The ‘plugin’ command tree on the module will provide commands for:
List installed repositories Add a new repository Remove a repository List installed plugins List/Search available plugins Install a plugin Remove a plugin Update a plugin
I would like to try to unify the web and script plugin systems a bit more closely. They are really two sides of the same coin (with some plugins providing both).
Does that make sense? Any suggestions/comments?
Regards, Mark.
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26 _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
Hi, I want to add my 3 plugins to the default repo. But could not find an example you were talking about. Can you send a direct link? Regards, Jaunius
Sent: Monday, September 28, 2020 at 9:59 AM From: "Mark Webb-Johnson" <mark@webb-johnson.net> To: "OVMS Developers" <ovmsdev@lists.openvehicles.com> Subject: Re: [Ovmsdev] Plugin Repository
The for-v3.3 branch now has a simple working plugin repository. While it can download assets of all types (json, module, web_page, web_hook, and web_rsc), it currently only supports installation and enable/disable for scripts.
Some notes:
Repositories are used to store plugins. The repository has a name and a url, and is accessed over http/https for download. A repository ‘openvehicles’ at 'http://api.openvehicles.com/plugins’ is installed by default. That repository published the OVMS v3 firmware plugins from our GitHub repository.
Repositories need to publish ‘plugins.json’ and ‘plugins.rev’ files in the root of their hosted directory. The first is JSON array of all the plugins hosted, and the second is a version number string for the repository. I’ve added a sample script to build the .json and .rev files from GitHub, and publish appropriately, in vehicle/OVMS.V3/support/buildplugins.pl.
You can add custom repositories with ‘plugin repo install’, and remove with ‘plugin repo remove’. The repository meta data is cached in RAM whenever a repository command is run (or you can force a refresh with ‘plugin repo refresh’.
OVMS# plugin repo list Repository Version openvehicles 3.2.014-98-g17b7989
Available plugins can be shown with ‘plugin list’.
OVMS# plugin list Plugin Repo Version Description chgthrottle openvehicles 1.2 Throttle charge current and/or stop charge if charger gets too hot. helloworld openvehicles 0.1 This plugin is provided for demonstration and example purposes only.
You can use ‘plugin install <name>’ to install a plugin, and ‘plugin remove <name>’ to remove it. The system will automatically download the plugin from the repository for you.
Plugins can be enabled/disabled with ‘plugin enable <name>’ and ‘plugin disable <name>’, and updated with ‘plugin update {<name>}’ (if <name> is ommitted, all plugins are updated).
Installing and enabling a plugin that provides javascript modules will now result in those modules automatically being loaded at duktape initialisation time. You don’t need to add them to ovmsmain.js.
Both wifi and cellular network connections are supported.
There is a ‘helloworld’ plugin now available as an example.
OVMS# plugin install helloworld Plugin: helloworld installed ok I (620571) pluginstore: Downloading plugin: helloworld, with 2 element(s) I (621671) pluginstore: Element: helloworld.js downloaded ok I (622441) pluginstore: Element: helloworld.json downloaded ok I (622441) pluginstore: Plugin: helloworld downloaded ok
OVMS# script reload Reloading javascript engine I (635121) ovms-duktape: Duktape: Clearing existing context I (635251) ovms-duktape: Duktape: Creating heap I (635351) ovms-duktape: Duktape: Initialising module system I (635601) pluginstore: Loading enabled plugins I (635601) pluginstore: Load helloworld (0.1)
OVMS# script eval 'helloworld.info()' Hello world
Next on my list is adding support for installing, enabling, and disabling web plugins; but I need to understand better how they integrate before I handle that.
After that, I need to handle the dependency system, and this should be done.
Regards, Mark.
On 16 Jul 2020, at 8:52 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Thanks, all, for the feedback.
@Nikolay: tags make sense and we can add. I think that in general the json could contain other fields (comments, notes, anything). The system itself really just needs name+version, with everything else as decoration or an enhancement to help searching.
@Derek: I’ll see how it looks when I upload. For download, a single zip is certainly easier, and this is not hard to do as part of the server script to publish a repository. Just not sure if it is actually necessary for us. Thanks for the link to the kodi repository structure.
@Michael: I think ‘module’ is the correct word (rather than ’script'. Let’s stick with that. I’ll work on this module side first, as I am not so familiar with the web extensions. The reason I suggested the ‘plugins’ directory (rather than ‘plugin’) was to allow us to move to this on an individual plugin basis. I’m not sure if that is necessary (we may instead be able to work off the existence of the json master file). I plan to try to make one plugin json and format first, then sort out the server side repository structure for that. Then we can review.
Regards, Mark.
On 14 Jul 2020, at 7:45 PM, Michael Balzer <dexter@expeedo.de <mailto:dexter@expeedo.de>> wrote:
Mark,
full ack, has been on my list as well. I'll handle the web UI side.
I don't think we need to migrate/change the folder structure, we should be able to just add the meta data. I tried to define the structure to be usable as a public repository. Did I miss something?
Btw, I used the naming "module" plugins instead of "script" plugins, as web plugins also consist mostly of scripts. I thought that would clear things for users, but I'm not sure now.
Regarding the plugin structure and meta data, we should decide wether to split plugins into their elements or keep them bundled. You seem to opt for bundles ("web subdirectory"). I would second that, but we then need to take into account a plugin may consist of multiple module and/or web plugins (see Edimax plugin for an example). Normally it's just one module plugin, but we should not restrict the system in that way. So the JSON file should allow to define an array of plugin elements / options.
Currently there's a version on each element, and but we could/should simplify this to a central overall version for the plugin.
To automatically install plugins, the meta data also needs to define the types and default setups for web/frontend plugins.
Draft: edimax.json:
{ name: "edimax", title: "Edimax: Smart Plug Control", description: "Smart Plug control for Edimax models SP-1101W, SP-2101W et al", group: "Home Automation", info: "https://docs.openvehicles.com/en/latest/plugin/edimax/README.html" <https://docs.openvehicles.com/en/latest/plugin/edimax/README.html>, maintainer: "Michael Balzer <dexter@dexters-web.de> <mailto:dexter@dexters-web.de>", version: "2.1", prerequisites: ["ovms>=3.2.010"], elements: [{ type: "script", path: "edimax.js", name: "edimax" // optional, defaults to plugin name },{ type: "webpage", path: "edimax.htm", page: "/usr/edimax", label: "Edimax Smart Plug", menu: "Config", auth: "Cookie" },{ type: "webhook", path: "edimax-status.htm", page: "/status", hook: "body.post" }] }
A method to easily disable/enable installed plugins would also be nice, i.e. as subcommands "plugin disable" / "plugin enable".
Regards, Michael
Am 14.07.20 um 07:55 schrieb Mark Webb-Johnson:
I feel that our plugin system offers a missed opportunity. By making small extensions to available functionality, we can use this system to do so much - and with a much lower complexity and barrier to entry compared to C++ ESP IDF development.
There are a couple of things I would like to now do:
Implement a mechanism for plugins to directly extend the console command language (other than the current ’script exec PLUGIN.fn’).
Implement a plugin repository, and command line tools for managing plugins. We simply need to have a store and directory of available plugins, a way of searching/listing those, and a uniform way to install/remove plugins. Once this is done, perhaps someone can handle the web UI side of this, and perhaps that can even be done as a plugin :-)
I’m already working on #1. Just an extension to the existing script system to allow plugins to register commands, and to have the command register/deregister done automatically on plugin load/unload.
However, #2 is a little more tricky and I’m looking for feedback on the design of this. My idea is:
Store the plugins in a new github/openvehicles/Open-Vehicle-Monitoring-System-3/plugins folder. Eventually the /plugin folder will all be migrated to here.
Each plugin has a unique name; the directory in /plugins.
Within each plugin directory, there is a <name>.json file (JSON format) with defined tags to describe the plugin. If the plugin delivers a web extension, that is in the ‘web’ subdirectory. If the plugin delivered a script extension, that is in the ’script’ directory. Documentation (in rst format) is in the ‘docs' directory. The JSON file would need the following, at a minimum:
Name: The plugin ID Title: One line title Group: Group for the plugin Version: The version string of the plugin Description: Longer textual description of the plugin Prerequisites: Prerequisites to be checked (Each plugin installed provides name=version, the system provides system-level ones such as the version of firmware, and then the prerequisites is simple a list of <name><operator><version> as required)
We can automatically build the documentation, and list, for all plugins for our docs.openvehicles.com <http://docs.openvehicles.com/> (in a similar way to how it is done for /plugin now).
On a repository server, as part of a nightly build job, we can automatically build a single directory JSON file from the <name>,json descriptors. We can publish this, as well as a git export/checkout of the plugins, on a http download site.
On the module, we can store a list of repository servers (allowing the user to add/remove as they require).
Contributors/authors of plugins simply GitHub clone our repository, and issue pull requests as appropriate. They can also have their own private repository servers (just need to host the directory json file and associated code files) and add it to their modules.
The ‘plugin’ command tree on the module will provide commands for:
List installed repositories Add a new repository Remove a repository List installed plugins List/Search available plugins Install a plugin Remove a plugin Update a plugin
I would like to try to unify the web and script plugin systems a bit more closely. They are really two sides of the same coin (with some plugins providing both).
Does that make sense? Any suggestions/comments?
Regards, Mark.
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26 _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
The examples are in the plugins directory of for-v3.3branch: https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/tree/for-v3... <https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/tree/for-v3.3/plugins> Regards, Mark
On 19 Oct 2020, at 3:55 PM, Jaunius Kapkan <jaunius@gmx.com> wrote:
Hi,
I want to add my 3 plugins to the default repo. But could not find an example you were talking about. Can you send a direct link?
Regards,
Jaunius
Sent: Monday, September 28, 2020 at 9:59 AM From: "Mark Webb-Johnson" <mark@webb-johnson.net> To: "OVMS Developers" <ovmsdev@lists.openvehicles.com> Subject: Re: [Ovmsdev] Plugin Repository
The for-v3.3 branch now has a simple working plugin repository. While it can download assets of all types (json, module, web_page, web_hook, and web_rsc), it currently only supports installation and enable/disable for scripts.
Some notes:
Repositories are used to store plugins. The repository has a name and a url, and is accessed over http/https for download. A repository ‘openvehicles’ at 'http://api.openvehicles.com/plugins’ is installed by default. That repository published the OVMS v3 firmware plugins from our GitHub repository.
Repositories need to publish ‘plugins.json’ and ‘plugins.rev’ files in the root of their hosted directory. The first is JSON array of all the plugins hosted, and the second is a version number string for the repository. I’ve added a sample script to build the .json and .rev files from GitHub, and publish appropriately, in vehicle/OVMS.V3/support/buildplugins.pl.
You can add custom repositories with ‘plugin repo install’, and remove with ‘plugin repo remove’. The repository meta data is cached in RAM whenever a repository command is run (or you can force a refresh with ‘plugin repo refresh’.
OVMS# plugin repo list Repository Version openvehicles 3.2.014-98-g17b7989
Available plugins can be shown with ‘plugin list’.
OVMS# plugin list Plugin Repo Version Description chgthrottle openvehicles 1.2 Throttle charge current and/or stop charge if charger gets too hot. helloworld openvehicles 0.1 This plugin is provided for demonstration and example purposes only.
You can use ‘plugin install <name>’ to install a plugin, and ‘plugin remove <name>’ to remove it. The system will automatically download the plugin from the repository for you.
Plugins can be enabled/disabled with ‘plugin enable <name>’ and ‘plugin disable <name>’, and updated with ‘plugin update {<name>}’ (if <name> is ommitted, all plugins are updated).
Installing and enabling a plugin that provides javascript modules will now result in those modules automatically being loaded at duktape initialisation time. You don’t need to add them to ovmsmain.js.
Both wifi and cellular network connections are supported.
There is a ‘helloworld’ plugin now available as an example.
OVMS# plugin install helloworld Plugin: helloworld installed ok I (620571) pluginstore: Downloading plugin: helloworld, with 2 element(s) I (621671) pluginstore: Element: helloworld.js downloaded ok I (622441) pluginstore: Element: helloworld.json downloaded ok I (622441) pluginstore: Plugin: helloworld downloaded ok
OVMS# script reload Reloading javascript engine I (635121) ovms-duktape: Duktape: Clearing existing context I (635251) ovms-duktape: Duktape: Creating heap I (635351) ovms-duktape: Duktape: Initialising module system I (635601) pluginstore: Loading enabled plugins I (635601) pluginstore: Load helloworld (0.1)
OVMS# script eval 'helloworld.info()' Hello world
Next on my list is adding support for installing, enabling, and disabling web plugins; but I need to understand better how they integrate before I handle that.
After that, I need to handle the dependency system, and this should be done.
Regards, Mark.
On 16 Jul 2020, at 8:52 AM, Mark Webb-Johnson <mark@webb-johnson.net> wrote:
Thanks, all, for the feedback.
@Nikolay: tags make sense and we can add. I think that in general the json could contain other fields (comments, notes, anything). The system itself really just needs name+version, with everything else as decoration or an enhancement to help searching.
@Derek: I’ll see how it looks when I upload. For download, a single zip is certainly easier, and this is not hard to do as part of the server script to publish a repository. Just not sure if it is actually necessary for us. Thanks for the link to the kodi repository structure.
@Michael: I think ‘module’ is the correct word (rather than ’script'. Let’s stick with that. I’ll work on this module side first, as I am not so familiar with the web extensions. The reason I suggested the ‘plugins’ directory (rather than ‘plugin’) was to allow us to move to this on an individual plugin basis. I’m not sure if that is necessary (we may instead be able to work off the existence of the json master file). I plan to try to make one plugin json and format first, then sort out the server side repository structure for that. Then we can review.
Regards, Mark.
On 14 Jul 2020, at 7:45 PM, Michael Balzer <dexter@expeedo.de <mailto:dexter@expeedo.de>> wrote:
Mark,
full ack, has been on my list as well. I'll handle the web UI side.
I don't think we need to migrate/change the folder structure, we should be able to just add the meta data. I tried to define the structure to be usable as a public repository. Did I miss something?
Btw, I used the naming "module" plugins instead of "script" plugins, as web plugins also consist mostly of scripts. I thought that would clear things for users, but I'm not sure now.
Regarding the plugin structure and meta data, we should decide wether to split plugins into their elements or keep them bundled. You seem to opt for bundles ("web subdirectory"). I would second that, but we then need to take into account a plugin may consist of multiple module and/or web plugins (see Edimax plugin for an example). Normally it's just one module plugin, but we should not restrict the system in that way. So the JSON file should allow to define an array of plugin elements / options.
Currently there's a version on each element, and but we could/should simplify this to a central overall version for the plugin.
To automatically install plugins, the meta data also needs to define the types and default setups for web/frontend plugins.
Draft: edimax.json:
{ name: "edimax", title: "Edimax: Smart Plug Control", description: "Smart Plug control for Edimax models SP-1101W, SP-2101W et al", group: "Home Automation", info: "https://docs.openvehicles.com/en/latest/plugin/edimax/README.html" <https://docs.openvehicles.com/en/latest/plugin/edimax/README.html>, maintainer: "Michael Balzer <dexter@dexters-web.de> <mailto:dexter@dexters-web.de>", version: "2.1", prerequisites: ["ovms>=3.2.010"], elements: [{ type: "script", path: "edimax.js", name: "edimax" // optional, defaults to plugin name },{ type: "webpage", path: "edimax.htm", page: "/usr/edimax", label: "Edimax Smart Plug", menu: "Config", auth: "Cookie" },{ type: "webhook", path: "edimax-status.htm", page: "/status", hook: "body.post" }] }
A method to easily disable/enable installed plugins would also be nice, i.e. as subcommands "plugin disable" / "plugin enable".
Regards, Michael
Am 14.07.20 um 07:55 schrieb Mark Webb-Johnson:
I feel that our plugin system offers a missed opportunity. By making small extensions to available functionality, we can use this system to do so much - and with a much lower complexity and barrier to entry compared to C++ ESP IDF development.
There are a couple of things I would like to now do:
Implement a mechanism for plugins to directly extend the console command language (other than the current ’script exec PLUGIN.fn’).
Implement a plugin repository, and command line tools for managing plugins. We simply need to have a store and directory of available plugins, a way of searching/listing those, and a uniform way to install/remove plugins. Once this is done, perhaps someone can handle the web UI side of this, and perhaps that can even be done as a plugin :-)
I’m already working on #1. Just an extension to the existing script system to allow plugins to register commands, and to have the command register/deregister done automatically on plugin load/unload.
However, #2 is a little more tricky and I’m looking for feedback on the design of this. My idea is:
Store the plugins in a new github/openvehicles/Open-Vehicle-Monitoring-System-3/plugins folder. Eventually the /plugin folder will all be migrated to here.
Each plugin has a unique name; the directory in /plugins.
Within each plugin directory, there is a <name>.json file (JSON format) with defined tags to describe the plugin. If the plugin delivers a web extension, that is in the ‘web’ subdirectory. If the plugin delivered a script extension, that is in the ’script’ directory. Documentation (in rst format) is in the ‘docs' directory. The JSON file would need the following, at a minimum:
Name: The plugin ID Title: One line title Group: Group for the plugin Version: The version string of the plugin Description: Longer textual description of the plugin Prerequisites: Prerequisites to be checked (Each plugin installed provides name=version, the system provides system-level ones such as the version of firmware, and then the prerequisites is simple a list of <name><operator><version> as required)
We can automatically build the documentation, and list, for all plugins for our docs.openvehicles.com <http://docs.openvehicles.com/> (in a similar way to how it is done for /plugin now).
On a repository server, as part of a nightly build job, we can automatically build a single directory JSON file from the <name>,json descriptors. We can publish this, as well as a git export/checkout of the plugins, on a http download site.
On the module, we can store a list of repository servers (allowing the user to add/remove as they require).
Contributors/authors of plugins simply GitHub clone our repository, and issue pull requests as appropriate. They can also have their own private repository servers (just need to host the directory json file and associated code files) and add it to their modules.
The ‘plugin’ command tree on the module will provide commands for:
List installed repositories Add a new repository Remove a repository List installed plugins List/Search available plugins Install a plugin Remove a plugin Update a plugin
I would like to try to unify the web and script plugin systems a bit more closely. They are really two sides of the same coin (with some plugins providing both).
Does that make sense? Any suggestions/comments?
Regards, Mark.
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev <http://lists.openvehicles.com/mailman/listinfo/ovmsdev>
-- Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal Fon 02333 / 833 5735 * Handy 0176 / 206 989 26 _______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com <mailto:OvmsDev@lists.openvehicles.com> http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
_______________________________________________ OvmsDev mailing list OvmsDev@lists.openvehicles.com http://lists.openvehicles.com/mailman/listinfo/ovmsdev
participants (4)
-
Jaunius Kapkan -
Mark Webb-Johnson -
Michael Balzer -
Nikolay Shishkov