CubicWeb

The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. The previous roadmap meeting was in January 2015. Christophe de Vienne (Unlish) and Aurélien Campéas (self-employed) joined us. Christophe de Vienne ... [More] asked for discussions on: Security Context: settle on an approach, and make it happen. Pyramid Cubicweb adoption: where are we? what authentication stack do we want by default? Package layout (aka "develop mode" friendliness): let's get real Documentation: is the restructuration attempt (https://www.cubicweb.org/ticket/4832808) a credible path for the documentation? Aurélien Campéas asked for discussions on: status of integration in the 3.21 branch a new API for cubicweb stores Sylvain Thénault asked for discussions on: a new API for dataimport (including cubicweb stores, but not only), new integrators on CW Versions Cubicweb Version 3.18 This version is stable but old and maintained (current is 3.18.8). Version 3.19 This version is stable and maintained (current is 3.19.9). Version 3.20 This version is now stable and maintained (current is 3.20.4). Version 3.21 See below Cubes cubicweb-bootstrap 0.6.6, 1.0.0, 1.0.1, 1.0.2 cubicweb-ckanpublish 0.1.0, 0.2.0 cubicweb-comment 1.11.0 cubicweb-company 0.6.1 cubicweb-forge 1.11.0 cubicweb-forgotpwd 0.6.2 cubicweb-iprogress 0.2.0 cubicweb-mandrill 0.4.0 cubicweb-nosylist 0.6.0 cubicweb-preview 1.1.1 cubicweb-registration 0.6.1 cubicweb-slickgrid 1.0.0 cubicweb-squareui 0.3.8, 1.0.0 cubicweb-testcard 0.5.0 cubicweb-tracker 1.16.2, 1.16.3 cubicweb-trackervcs 1.3.0 cubicweb-trustedauth 0.3.1 cubicweb-varnish 0.3.0 cubicweb-vcreview 2.1.0, 2.1.1 cubicweb-vcsfile 2.0.1, 2.0.2, 2.0.3 cubicweb-worker 3.0.5 cubicweb-wsme 0.1.6 narval 4.1.2 pyramid-cubicweb 0.2.0, 0.2.1 Agenda Next roadmap meeting will be held at the beginning of may 2015 at Logilab. Interested parties are invited to get in touch. Open Discussions New integrators Rémi Cardona (rcardona) and Denis Laxaldle (dlaxalde) have now the publish access level on Cubicweb repositories. Security context Christophe exposed his proposal for a "security context" in Cubicweb, as exposed in https://lists.cubicweb.org/pipermail/cubicweb/2015-February/002278.html and https://lists.cubicweb.org/pipermail/cubicweb/2015-February/002297.html with a proposition of implementation (see https://www.cubicweb.org/ticket/4919855 ) The idea has been validated based on a substitution variables, which names will start with "ctx:" (the RQL grammar will have to be modified to accept a ":") This will then allow to write RQL queries like (API still to be tuned): X owned_by U, U eid %(ctx:cwuser_eid)s Pyramid The pyramid-based web server proposed by Christophe and used for its unlish website is still under test and evaluation at Logilab. There are missing features (implemented in cubes) required to be able to deploy pyramid-cubicweb for most of the applications used at Logilab, especially cubicweb-signedrequest In order to make it possible to implement authentication cubes like cubicweb-signedrequest, the pyramid-cubicweb requires some modifications. These has been developped and are about to be published, along with a new version of signedrequest that provide pyramid compatibility. There are still some dependencies that lack a proper Debian package, but that should be done in the next few weeks. In order to properly identify pyramid-related code in a cube, it has been proposed that these code should go in modules in the cube named pviews and pconfig (note that most cube won't require any pyramid specific code). The includeme function should however be in the cube's main packgage (in the __init__.py file) There have been some discussions about the fact that, for now, a pyramid-cubicweb instance requires an anonymous user/access, which can also be a problem for some application. Layout Christophe pointed the fact that the directory/files layout of cubicweb and cubes do not follow current Python's de facto standards, which makes cubicweb hard to use in a context of virtualenv/pip based installation. There is the CWEP004 discussing some aspects of this problem. The decision has been taken to move toward a Cubicweb ecosystem that is more pip-friendly. This will be done step by step, starting with the dependencies (packages currently living in the logilab "namespace"). Then we will investigate the feasibility of migrating the layout of Cubicweb itself. Documentation The new documentation structure has been approved. It has been proposed (and more or less accepted) to extract the documentation in a dedicated project. This is not a priority, however. Roadmap for 3.21 No change since last meeting: the complete removal of the dbapi, the merging of Connection and ClientConnection. remains Integrate the pyramid cube to provide the pyramid command if the pyramid framework can be imported: removed (too soon, pyramid-cubicweb's APIs are not stable enough) Integration of CWEP-003 (FROM clause for RQL): removed (will probably never be included unless someone needs it) CWEP-004 (cubes as standard python packages) is being discussed: removed (not for 3.21, see above) dataimports et stores A heavy refactoring is under way that concerns data import in CubicWeb. The main goal is to design a single API to be used by the various cubes that accelerate the insertion of data (dataio, massiveimport, fastimport, etc) as well as the internal CWSource and its data feeds. For details, see the thread on the mailing-list and the patches arriving in the review pipeline. [Less]

The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. The previous roadmap meeting was in January 2015. Christophe de Vienne (Unlish) and Aurélien Campéas (self-employed) joined us. Christophe de Vienne ... [More] asked for discussions on: Security Context: settle on an approach, and make it happen. Pyramid Cubicweb adoption: where are we? what authentication stack do we want by default? Package layout (aka "develop mode" friendliness): let's get real Documentation: is the restructuration attempt (https://www.cubicweb.org/ticket/4832808) a credible path for the documentation? Aurélien Campéas asked for discussions on: status of integration in the 3.21 branch a new API for cubicweb stores Sylvain Thénault asked for discussions on: a new API for dataimport (including cubicweb stores, but not only), new integrators on CW Versions Cubicweb Version 3.18 This version is stable but old and maintained (current is 3.18.8). Version 3.19 This version is stable and maintained (current is 3.19.9). Version 3.20 This version is now stable and maintained (current is 3.20.4). Version 3.21 See below Cubes cubicweb-bootstrap 0.6.6, 1.0.0, 1.0.1, 1.0.2 cubicweb-ckanpublish 0.1.0, 0.2.0 cubicweb-comment 1.11.0 cubicweb-company 0.6.1 cubicweb-forge 1.11.0 cubicweb-forgotpwd 0.6.2 cubicweb-iprogress 0.2.0 cubicweb-mandrill 0.4.0 cubicweb-nosylist 0.6.0 cubicweb-preview 1.1.1 cubicweb-registration 0.6.1 cubicweb-slickgrid 1.0.0 cubicweb-squareui 0.3.8, 1.0.0 cubicweb-testcard 0.5.0 cubicweb-tracker 1.16.2, 1.16.3 cubicweb-trackervcs 1.3.0 cubicweb-trustedauth 0.3.1 cubicweb-varnish 0.3.0 cubicweb-vcreview 2.1.0, 2.1.1 cubicweb-vcsfile 2.0.1, 2.0.2, 2.0.3 cubicweb-worker 3.0.5 cubicweb-wsme 0.1.6 narval 4.1.2 pyramid-cubicweb 0.2.0, 0.2.1 Agenda Next roadmap meeting will be held at the beginning of may 2015 at Logilab. Interested parties are invited to get in touch. Open Discussions New integrators Rémi Cardona (rcardona) and Denis Laxaldle (dlaxalde) have now the publish access level on Cubicweb repositories. Security context Christophe exposed his proposal for a "security context" in Cubicweb, as exposed in https://lists.cubicweb.org/pipermail/cubicweb/2015-February/002278.html and https://lists.cubicweb.org/pipermail/cubicweb/2015-February/002297.html with a proposition of implementation (see https://www.cubicweb.org/ticket/4919855 ) The idea has been validated based on a substitution variables, which names will start with "ctx:" (the RQL grammar will have to be modified to accept a ":") This will then allow to write RQL queries like (API still to be tuned): X owned_by U, U eid %(ctx:cwuser_eid)s Pyramid The pyramid-based web server proposed by Christophe and used for its unlish website is still under test and evaluation at Logilab. There are missing features (implemented in cubes) required to be able to deploy pyramid-cubicweb for most of the applications used at Logilab, especially cubicweb-signedrequest In order to make it possible to implement authentication cubes like cubicweb-signedrequest, the pyramid-cubicweb requires some modifications. These has been developped and are about to be published, along with a new version of signedrequest that provide pyramid compatibility. There are still some dependencies that lack a proper Debian package, but that should be done in the next few weeks. In order to properly identify pyramid-related code in a cube, it has been proposed that these code should go in modules in the cube named pviews and pconfig (note that most cube won't require any pyramid specific code). The includeme function should however be in the cube's main packgage (in the __init__.py file) There have been some discussions about the fact that, for now, a pyramid-cubicweb instance requires an anonymous user/access, which can also be a problem for some application. Layout Christophe pointed the fact that the directory/files layout of cubicweb and cubes do not follow current Python's de facto standards, which makes cubicweb hard to use in a context of virtualenv/pip based installation. There is the CWEP004 discussing some aspects of this problem. The decision has been taken to move toward a Cubicweb ecosystem that is more pip-friendly. This will be done step by step, starting with the dependencies (packages currently living in the logilab "namespace"). Then we will investigate the feasibility of migrating the layout of Cubicweb itself. Documentation The new documentation structure has been approved. It has been proposed (and more or less accepted) to extract the documentation in a dedicated project. This is not a priority, however. Roadmap for 3.21 No change since last meeting: the complete removal of the dbapi, the merging of Connection and ClientConnection. remains Integrate the pyramid cube to provide the pyramid command if the pyramid framework can be imported: removed (too soon, pyramid-cubicweb's APIs are not stable enough) Integration of CWEP-003 (FROM clause for RQL): removed (will probably never be included unless someone needs it) CWEP-004 (cubes as standard python packages) is being discussed: removed (not for 3.21, see above) dataimports et stores A heavy refactoring is under way that concerns data import in CubicWeb. The main goal is to design a single API to be used by the various cubes that accelerate the insertion of data (dataio, massiveimport, fastimport, etc) as well as the internal CWSource and its data feeds. For details, see the thread on the mailing-list and the patches arriving in the review pipeline. [Less]

The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. The previous roadmap meeting was in January 2015. Christophe de Vienne (Unlish) and Aurélien Campéas (self-employed) joined us. Christophe de Vienne ... [More] asked for discussions on: Security Context: settle on an approach, and make it happen. Pyramid Cubicweb adoption: where are we? what authentication stack do we want by default? Package layout (aka "develop mode" friendliness): let's get real Documentation: is the restructuration attempt (https://www.cubicweb.org/ticket/4832808) a credible path for the documentation? Aurélien Campéas asked for discussions on: status of integration in the 3.21 branch a new API for cubicweb stores Sylvain Thénault asked for discussions on: a new API for dataimport (including cubicweb stores, but not only), new integrators on CW Versions Cubicweb Version 3.18 This version is stable but old and maintained (current is 3.18.8). Version 3.19 This version is stable and maintained (current is 3.19.9). Version 3.20 This version is now stable and maintained (current is 3.20.4). Version 3.21 See below Cubes cubicweb-bootstrap 0.6.6, 1.0.0, 1.0.1, 1.0.2 cubicweb-ckanpublish 0.1.0, 0.2.0 cubicweb-comment 1.11.0 cubicweb-company 0.6.1 cubicweb-forge 1.11.0 cubicweb-forgotpwd 0.6.2 cubicweb-iprogress 0.2.0 cubicweb-mandrill 0.4.0 cubicweb-nosylist 0.6.0 cubicweb-preview 1.1.1 cubicweb-registration 0.6.1 cubicweb-slickgrid 1.0.0 cubicweb-squareui 0.3.8, 1.0.0 cubicweb-testcard 0.5.0 cubicweb-tracker 1.16.2, 1.16.3 cubicweb-trackervcs 1.3.0 cubicweb-trustedauth 0.3.1 cubicweb-varnish 0.3.0 cubicweb-vcreview 2.1.0, 2.1.1 cubicweb-vcsfile 2.0.1, 2.0.2, 2.0.3 cubicweb-worker 3.0.5 cubicweb-wsme 0.1.6 narval 4.1.2 pyramid-cubicweb 0.2.0, 0.2.1 Agenda Next roadmap meeting will be held at the beginning of may 2015 at Logilab. Interested parties are invited to get in touch. Open Discussions New integrators Rémi Cardona (rcardona) and Denis Laxaldle (dlaxalde) have now the publish access level on Cubicweb repositories. Security context Christophe exposed his proposal for a "security context" in Cubicweb, as exposed in https://lists.cubicweb.org/pipermail/cubicweb/2015-February/002278.html and https://lists.cubicweb.org/pipermail/cubicweb/2015-February/002297.html with a proposition of implementation (see https://www.cubicweb.org/ticket/4919855 ) The idea has been validated based on a substitution variables, which names will start with "ctx:" (the RQL grammar will have to be modified to accept a ":") This will then allow to write RQL queries like (API still to be tuned): X owned_by U, U eid %(ctx:cwuser_eid)s Pyramid The pyramid-based web server proposed by Christophe and used for its unlish website is still under test and evaluation at Logilab. There are missing features (implemented in cubes) required to be able to deploy pyramid-cubicweb for most of the applications used at Logilab, especially cubicweb-signedrequest In order to make it possible to implement authentication cubes like cubicweb-signedrequest, the pyramid-cubicweb requires some modifications. These has been developped and are about to be published, along with a new version of signedrequest that provide pyramid compatibility. There are still some dependencies that lack a proper Debian package, but that should be done in the next few weeks. In order to properly identify pyramid-related code in a cube, it has been proposed that these code should go in modules in the cube named pviews and pconfig (note that most cube won't require any pyramid specific code). The includeme function should however be in the cube's main packgage (in the __init__.py file) There have been some discussions about the fact that, for now, a pyramid-cubicweb instance requires an anonymous user/access, which can also be a problem for some application. Layout Christophe pointed the fact that the directory/files layout of cubicweb and cubes do not follow current Python's de facto standards, which makes cubicweb hard to use in a context of virtualenv/pip based installation. There is the CWEP004 discussing some aspects of this problem. The decision has been taken to move toward a Cubicweb ecosystem that is more pip-friendly. This will be done step by step, starting with the dependencies (packages currently living in the logilab "namespace"). Then we will investigate the feasibility of migrating the layout of Cubicweb itself. Documentation The new documentation structure has been approved. It has been proposed (and more or less accepted) to extract the documentation in a dedicated project. This is not a priority, however. Roadmap for 3.21 No change since last meeting: the complete removal of the dbapi, the merging of Connection and ClientConnection. remains Integrate the pyramid cube to provide the pyramid command if the pyramid framework can be imported: removed (too soon, pyramid-cubicweb's APIs are not stable enough) Integration of CWEP-003 (FROM clause for RQL): removed (will probably never be included unless someone needs it) CWEP-004 (cubes as standard python packages) is being discussed: removed (not for 3.21, see above) dataimports et stores A heavy refactoring is under way that concerns data import in CubicWeb. The main goal is to design a single API to be used by the various cubes that accelerate the insertion of data (dataio, massiveimport, fastimport, etc) as well as the internal CWSource and its data feeds. For details, see the thread on the mailing-list and the patches arriving in the review pipeline. [Less]

The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. The previous roadmap meeting was in November 2014. Here is the report about the January 8th, 2015 meeting. Christophe de Vienne (Unlish) and Aurélien ... [More] Campéas (self-employed) joined us to express their concerns and discuss the future of CubicWeb. Versions Version 3.18 This version is stable but old and maintained (current is 3.18.7). Version 3.19 This version is stable and maintained (current is 3.19.8). Version 3.20 This version has been released a few days ago. It has not been deployed on production systems yet. Its main features are: virtual relations: a new ComputedRelation class can be used in schema.py; its rule attribute is an RQL snippet that defines the new relation. computed attributes: an attribute can now be defined with a formula argument (also an RQL snippet); it will be read-only, and updated automatically. Both of these features are described in CWEP-002, and the updated "Data model" chapter of the CubicWeb book. cubicweb-ctl plugins can use the cubicweb.utils.admincnx function to get a Connection object from an instance name. new 'tornado' wsgi backend session cookies have the HttpOnly flag, so they're no longer exposed to javascript rich text fields can be formatted as markdown the edit controller detects concurrent editions, and raises a ValidationError if an entity was modified between form generation and submission cubicweb can use a postgresql "schema" (namespace) for its tables cubicweb-ctl configure can be used to set values of the admin user credentials in the sources configuration file For details read list of tickets for CubicWeb 3.20.0. We would have loved to integrate the pyramid cube in this release, but the debian packaging effort needed by the pyramid stack is quite big and is acceptable if we target jessie only (at decent price). Version 3.21 For now, the roadmap for 3.21 is still the complete removal of the dbapi, the merging of Connection and ClientConnection. Integrate the pyramid cube to provide the pyramid command if the pyramid framework can be imported. Integration of CWEP-003 (FROM clause for RQL) and CWEP-004 (cubes as standard python packages) is being discussed. Version 4.0 We expect to accelerate development of CubicWeb 4, which exact roadmap is still to be discussed, but we may already want: be pyramid-based (remove twisted, auth management, etc.), do not have anything left of old dbapi and ClientConnection, integrate squareui as main (and only) web-ui "template" or remove web generation (almost) completely from cubicweb-core and provide it only through the cube system. Cubes apycot 3.2.0 cubicweb-brainomics 0.11.5 cubicweb-card 0.5.4 cubicweb-container 2.7.0 cubicweb-elections 0.2.0, 0.2.1 cubicweb-fastimport 0.2.1 cubicweb-forgotpwd 0.6.1 cubicweb-inlinedit 1.2.1 cubicweb-mercurial-server 0.6.0 cubicweb-nazcaui 0.3.1 cubicweb-registration 0.6.0 cubicweb-signedrequest 0.1.3 cubicweb-subprocess 0.2.1 cubicweb-timeseries 1.4.0 cubicweb-timesheet 0.12.0 cubicweb-trackervcs 1.2.0 cubicweb-transactionlog 0.3.0 cubicweb-treeview 0.1.1 cubicweb-vcreview 2.0.0 cubicweb-vcsfile 1.17.0, 2.0.0 cubicweb-vcwiki 0.3.0 cubicweb-worker 3.1.0 cubicweb-wsme 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5 narval 4.1.1 cwclientlib 0.2.1 pyramid-cubicweb 0.1.2, 0.1.3 rqlquery 0.2.0, 0.2.1 Agenda Next roadmap meeting will be held at the beginning of march 2015 at Logilab. Interested parties are invited to get in touch. Open Discussions Refactoring the documentation Christophe de Vienne suggested to completely revamp the documentation and intends to lead this effort. Training material Aurélien Campéas asks if Logilab would be willing to share its training material under a free license to help interested parties organize and sell trainings. Towards making squareui the default rendering engine for cubicweb We are expecting to be able to use squareui/bootstrap as "rendering engine" for our forge applications (like http://www.cubicweb.org and http://www.logilab.org) as soon as possible. However to achieve to goal, there are still too many "visual bugs", some of which may require a discussion. Among others: put the ctxtoolbar component in the <nav> div each box component should have an icon (what API for this?) we cannot easily make the left column of the main template responsive-aware (requires to change the html flow), so it's probably best to take inspiration from things like http://wrapbootstrap.com/preview/WB0N89JMK facet boxes are a mess, there is no simple solution to have a "smart layout" Migration AppObjects should not be loaded by default Have a look at Alembic the migration tool for SQLAlchemy and take inspiration from there. [Less]

The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. The previous roadmap meeting was in November 2014. Here is the report about the January 8th, 2015 meeting. Christophe de Vienne (Unlish) and Aurélien ... [More] Campéas (self-employed) joined us to express their concerns and discuss the future of CubicWeb. Versions Version 3.18 This version is stable but old and maintained (current is 3.18.7). Version 3.19 This version is stable and maintained (current is 3.19.8). Version 3.20 This version has been released a few days ago. It has not been deployed on production systems yet. Its main features are: virtual relations: a new ComputedRelation class can be used in schema.py; its rule attribute is an RQL snippet that defines the new relation. computed attributes: an attribute can now be defined with a formula argument (also an RQL snippet); it will be read-only, and updated automatically. Both of these features are described in CWEP-002, and the updated "Data model" chapter of the CubicWeb book. cubicweb-ctl plugins can use the cubicweb.utils.admincnx function to get a Connection object from an instance name. new 'tornado' wsgi backend session cookies have the HttpOnly flag, so they're no longer exposed to javascript rich text fields can be formatted as markdown the edit controller detects concurrent editions, and raises a ValidationError if an entity was modified between form generation and submission cubicweb can use a postgresql "schema" (namespace) for its tables cubicweb-ctl configure can be used to set values of the admin user credentials in the sources configuration file For details read list of tickets for CubicWeb 3.20.0. We would have loved to integrate the pyramid cube in this release, but the debian packaging effort needed by the pyramid stack is quite big and is acceptable if we target jessie only (at decent price). Version 3.21 For now, the roadmap for 3.21 is still the complete removal of the dbapi, the merging of Connection and ClientConnection. Integrate the pyramid cube to provide the pyramid command if the pyramid framework can be imported. Integration of CWEP-003 (FROM clause for RQL) and CWEP-004 (cubes as standard python packages) is being discussed. Version 4.0 We expect to accelerate development of CubicWeb 4, which exact roadmap is still to be discussed, but we may already want: be pyramid-based (remove twisted, auth management, etc.), do not have anything left of old dbapi and ClientConnection, integrate squareui as main (and only) web-ui "template" or remove web generation (almost) completely from cubicweb-core and provide it only through the cube system. Cubes apycot 3.2.0 cubicweb-brainomics 0.11.5 cubicweb-card 0.5.4 cubicweb-container 2.7.0 cubicweb-elections 0.2.0, 0.2.1 cubicweb-fastimport 0.2.1 cubicweb-forgotpwd 0.6.1 cubicweb-inlinedit 1.2.1 cubicweb-mercurial-server 0.6.0 cubicweb-nazcaui 0.3.1 cubicweb-registration 0.6.0 cubicweb-signedrequest 0.1.3 cubicweb-subprocess 0.2.1 cubicweb-timeseries 1.4.0 cubicweb-timesheet 0.12.0 cubicweb-trackervcs 1.2.0 cubicweb-transactionlog 0.3.0 cubicweb-treeview 0.1.1 cubicweb-vcreview 2.0.0 cubicweb-vcsfile 1.17.0, 2.0.0 cubicweb-vcwiki 0.3.0 cubicweb-worker 3.1.0 cubicweb-wsme 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5 narval 4.1.1 cwclientlib 0.2.1 pyramid-cubicweb 0.1.2, 0.1.3 rqlquery 0.2.0, 0.2.1 Agenda Next roadmap meeting will be held at the beginning of march 2015 at Logilab. Interested parties are invited to get in touch. Open Discussions Refactoring the documentation Christophe de Vienne suggested to completely revamp the documentation and intends to lead this effort. Training material Aurélien Campéas asks if Logilab would be willing to share its training material under a free license to help interested parties organize and sell trainings. Towards making squareui the default rendering engine for cubicweb We are expecting to be able to use squareui/bootstrap as "rendering engine" for our forge applications (like http://www.cubicweb.org and http://www.logilab.org) as soon as possible. However to achieve to goal, there are still too many "visual bugs", some of which may require a discussion. Among others: put the ctxtoolbar component in the <nav> div each box component should have an icon (what API for this?) we cannot easily make the left column of the main template responsive-aware (requires to change the html flow), so it's probably best to take inspiration from things like http://wrapbootstrap.com/preview/WB0N89JMK facet boxes are a mess, there is no simple solution to have a "smart layout" Migration AppObjects should not be loaded by default Have a look at Alembic the migration tool for SQLAlchemy and take inspiration from there. [Less]

The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. The previous roadmap meeting was in November 2014. Here is the report about the January 8th, 2015 meeting. Christophe de Vienne (Unlish) and Aurélien ... [More] Campéas (self-employed) joined us to express their concerns and discuss the future of CubicWeb. Versions Version 3.18 This version is stable but old and maintained (current is 3.18.7). Version 3.19 This version is stable and maintained (current is 3.19.8). Version 3.20 This version has been released a few days ago. It has not been deployed on production systems yet. Its main features are: virtual relations: a new ComputedRelation class can be used in schema.py; its rule attribute is an RQL snippet that defines the new relation. computed attributes: an attribute can now be defined with a formula argument (also an RQL snippet); it will be read-only, and updated automatically. Both of these features are described in CWEP-002, and the updated "Data model" chapter of the CubicWeb book. cubicweb-ctl plugins can use the cubicweb.utils.admincnx function to get a Connection object from an instance name. new 'tornado' wsgi backend session cookies have the HttpOnly flag, so they're no longer exposed to javascript rich text fields can be formatted as markdown the edit controller detects concurrent editions, and raises a ValidationError if an entity was modified between form generation and submission cubicweb can use a postgresql "schema" (namespace) for its tables cubicweb-ctl configure can be used to set values of the admin user credentials in the sources configuration file For details read list of tickets for CubicWeb 3.20.0. We would have loved to integrate the pyramid cube in this release, but the debian packaging effort needed by the pyramid stack is quite big and is acceptable if we target jessie only (at decent price). Version 3.21 For now, the roadmap for 3.21 is still the complete removal of the dbapi, the merging of Connection and ClientConnection. Integrate the pyramid cube to provide the pyramid command if the pyramid framework can be imported. Integration of CWEP-003 (FROM clause for RQL) and CWEP-004 (cubes as standard python packages) is being discussed. Version 4.0 We expect to accelerate development of CubicWeb 4, which exact roadmap is still to be discussed, but we may already want: be pyramid-based (remove twisted, auth management, etc.), do not have anything left of old dbapi and ClientConnection, integrate squareui as main (and only) web-ui "template" or remove web generation (almost) completely from cubicweb-core and provide it only through the cube system. Cubes apycot 3.2.0 cubicweb-brainomics 0.11.5 cubicweb-card 0.5.4 cubicweb-container 2.7.0 cubicweb-elections 0.2.0, 0.2.1 cubicweb-fastimport 0.2.1 cubicweb-forgotpwd 0.6.1 cubicweb-inlinedit 1.2.1 cubicweb-mercurial-server 0.6.0 cubicweb-nazcaui 0.3.1 cubicweb-registration 0.6.0 cubicweb-signedrequest 0.1.3 cubicweb-subprocess 0.2.1 cubicweb-timeseries 1.4.0 cubicweb-timesheet 0.12.0 cubicweb-trackervcs 1.2.0 cubicweb-transactionlog 0.3.0 cubicweb-treeview 0.1.1 cubicweb-vcreview 2.0.0 cubicweb-vcsfile 1.17.0, 2.0.0 cubicweb-vcwiki 0.3.0 cubicweb-worker 3.1.0 cubicweb-wsme 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5 narval 4.1.1 cwclientlib 0.2.1 pyramid-cubicweb 0.1.2, 0.1.3 rqlquery 0.2.0, 0.2.1 Agenda Next roadmap meeting will be held at the beginning of march 2015 at Logilab. Interested parties are invited to get in touch. Open Discussions Refactoring the documentation Christophe de Vienne suggested to completely revamp the documentation and intends to lead this effort. Training material Aurélien Campéas asks if Logilab would be willing to share its training material under a free license to help interested parties organize and sell trainings. Towards making squareui the default rendering engine for cubicweb We are expecting to be able to use squareui/bootstrap as "rendering engine" for our forge applications (like http://www.cubicweb.org and http://www.logilab.org) as soon as possible. However to achieve to goal, there are still too many "visual bugs", some of which may require a discussion. Among others: put the ctxtoolbar component in the div each box component should have an icon (what API for this?) we cannot easily make the left column of the main template responsive-aware (requires to change the html flow), so it's probably best to take inspiration from things like http://wrapbootstrap.com/preview/WB0N89JMK facet boxes are a mess, there is no simple solution to have a "smart layout" Migration AppObjects should not be loaded by default Have a look at Alembic the migration tool for SQLAlchemy and take inspiration from there. [Less]

The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. The previous roadmap meeting was in September 2014. Here is the report about the November 6th, 2014 meeting. Christophe de Vienne (Unlish) joined us ... [More] to express their concerns and discuss the future of CubicWeb. Dimitri Papadopoulos (CEA) could not come. Versions Version 3.17 This version is stable but old and maintainance will continue only as long as some customers will be willing to pay for it (current is 3.17.17). If you're still using 3.17, you should go directly to 3.19. Version 3.18 This version is stable but old and maintained (current is 3.18.6). Version 3.19 This version is stable and maintained (current is 3.19.5). Version 3.20 This version is still under development but should be released very soon now (expected next week). Its main feature being the inclusion of CWEP-002 (computed attributes and relations), along with many small improvement patches. For details read list of tickets for CubicWeb 3.20.0. We would have loved to integrate the pyramid cube in this release, but the debian packaging effort needed by the pyramid stack is quite big and is acceptable if we target jessie only (at decent price). Version 3.21 For now, the roadmap for 3.21 is still the complete removal of the dbapi, the merging of Connection and ClientConnection, and possibly including CWEP-003 (adding a FROM clause to RQL). Integrate the pyramid cube to provide the pyramid command if the pyramid framework can be imported. Integration of CWEP-004 is being discussed. Version 4.0 We expect to accelerate development of CubicWeb 4, which exact roadmap is still to be discussed, but we may already want: be pyramid-based (remove twisted, auth management, etc.), do not have anything left of old dbapi and ClientConnection, integrate squareui as main (and only) web-ui "template" or remove web generation (almost) completely from cubicweb-core and provide it only through the cube system. Cubes cubicweb-bootstrap (0.6.5) cubicweb-brainomics (0.11.4, 0.12.0) cubicweb-clinipath (0.2.1, 0.2.2) cubicweb-collaboration (1.0.1) cubicweb-fastimport (0.2.0) cubicweb-file (1.16.1) cubicweb-forge (1.10.2) cubicweb-forgotpwd (0.6.0) cubicweb-invoice (0.8.0) cubicweb-leaflet (0.2.1, 0.3.0) cubicweb-mediaplayer (0.1.3, 0.1.4) cubicweb-medicalexp (0.12.3) cubicweb-osmfrance (0.2.2) cubicweb-person (1.8.1, 1.9.0) cubicweb-postgis (0.3.0) cubicweb-relationwidget (0.2.0) cubicweb-rqlcontroller (0.2.0, 0.3.0) cubicweb-squareui (0.3.7) cubicweb-subprocess (0.2.0) cubicweb-tracker (1.16.1) cubicweb-transactionlog (0.2.0, 0.2.1) cubicweb-worker (3.0.3, 3.0.4) cubicweb-wsme (0.1.0) cwclientlib (0.2.0) cwtags (1.1.0) New cubes and libraries cubicweb-i18nfield (0.1.0) cubicweb-pyramid (0.1.0) cubicweb-massmailing (0.1.0) cubicweb-wsme (0.1.0) pyramid-cubicweb (0.1.0, 0.1.1) rqlquery (0.1.0, 0.1.1) CWEPs Here is the status of open CubicWeb Evolution Proposals: to be written Work in progress Some work is in progress around CKAN, DCAT and othr Open Data and Semantic Web related technologies. Agenda Next roadmap meeting will be held at the beginning of january 2015 at Logilab, and Christophe and Dimitri (or Yann) are invited. Open Discussions Migration: AppObjects should not be loaded by default Have a look at Alembic the migration tool for SQLAlchemy and take inspiration from there [Less]

The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. The previous roadmap meeting was in September 2014. Here is the report about the November 6th, 2014 meeting. Christophe de Vienne (Unlish) joined us ... [More] to express their concerns and discuss the future of CubicWeb. Dimitri Papadopoulos (CEA) could not come. Versions Version 3.17 This version is stable but old and maintainance will continue only as long as some customers will be willing to pay for it (current is 3.17.17). If you're still using 3.17, you should go directly to 3.19. Version 3.18 This version is stable but old and maintained (current is 3.18.6). Version 3.19 This version is stable and maintained (current is 3.19.5). Version 3.20 This version is still under development but should be released very soon now (expected next week). Its main feature being the inclusion of CWEP-002 (computed attributes and relations), along with many small improvement patches. For details read list of tickets for CubicWeb 3.20.0. We would have loved to integrate the pyramid cube in this release, but the debian packaging effort needed by the pyramid stack is quite big and is acceptable if we target jessie only (at decent price). Version 3.21 For now, the roadmap for 3.21 is still the complete removal of the dbapi, the merging of Connection and ClientConnection, and possibly including CWEP-003 (adding a FROM clause to RQL). Integrate the pyramid cube to provide the pyramid command if the pyramid framework can be imported. Integration of CWEP-004 is being discussed. Version 4.0 We expect to accelerate development of CubicWeb 4, which exact roadmap is still to be discussed, but we may already want: be pyramid-based (remove twisted, auth management, etc.), do not have anything left of old dbapi and ClientConnection, integrate squareui as main (and only) web-ui "template" or remove web generation (almost) completely from cubicweb-core and provide it only through the cube system. Cubes cubicweb-bootstrap (0.6.5) cubicweb-brainomics (0.11.4, 0.12.0) cubicweb-clinipath (0.2.1, 0.2.2) cubicweb-collaboration (1.0.1) cubicweb-fastimport (0.2.0) cubicweb-file (1.16.1) cubicweb-forge (1.10.2) cubicweb-forgotpwd (0.6.0) cubicweb-invoice (0.8.0) cubicweb-leaflet (0.2.1, 0.3.0) cubicweb-mediaplayer (0.1.3, 0.1.4) cubicweb-medicalexp (0.12.3) cubicweb-osmfrance (0.2.2) cubicweb-person (1.8.1, 1.9.0) cubicweb-postgis (0.3.0) cubicweb-relationwidget (0.2.0) cubicweb-rqlcontroller (0.2.0, 0.3.0) cubicweb-squareui (0.3.7) cubicweb-subprocess (0.2.0) cubicweb-tracker (1.16.1) cubicweb-transactionlog (0.2.0, 0.2.1) cubicweb-worker (3.0.3, 3.0.4) cubicweb-wsme (0.1.0) cwclientlib (0.2.0) cwtags (1.1.0) New cubes and libraries cubicweb-i18nfield (0.1.0) cubicweb-pyramid (0.1.0) cubicweb-massmailing (0.1.0) cubicweb-wsme (0.1.0) pyramid-cubicweb (0.1.0, 0.1.1) rqlquery (0.1.0, 0.1.1) CWEPs Here is the status of open CubicWeb Evolution Proposals: to be written Work in progress Some work is in progress around CKAN, DCAT and othr Open Data and Semantic Web related technologies. Agenda Next roadmap meeting will be held at the beginning of january 2015 at Logilab, and Christophe and Dimitri (or Yann) are invited. Open Discussions Migration: AppObjects should not be loaded by default Have a look at Alembic the migration tool for SQLAlchemy and take inspiration from there [Less]

The Logilab team holds a roadmap meeting every two months to plan its CubicWeb development effort. The previous roadmap meeting was in September 2014. Here is the report about the November 6th, 2014 meeting. Christophe de Vienne (Unlish) joined us to ... [More] express their concerns and discuss the future of CubicWeb. Dimitri Papadopoulos (CEA) could not come. Versions Version 3.17 This version is stable but old and maintainance will continue only as long as some customers will be willing to pay for it (current is 3.17.17). If you're still using 3.17, you should go directly to 3.19. Version 3.18 This version is stable but old and maintained (current is 3.18.6). Version 3.19 This version is stable and maintained (current is 3.19.5). Version 3.20 This version is still under development but should be released very soon now (expected next week). Its main feature being the inclusion of CWEP-002 (computed attributes and relations), along with many small improvement patches. For details read list of tickets for CubicWeb 3.20.0. We would have loved to integrate the pyramid cube in this release, but the debian packaging effort needed by the pyramid stack is quite big and is acceptable if we target jessie only (at decent price). Version 3.21 For now, the roadmap for 3.21 is still the complete removal of the dbapi, the merging of Connection and ClientConnection, and possibly including CWEP-003 (adding a FROM clause to RQL). Integrate the pyramid cube to provide the pyramid command if the pyramid framework can be imported. Integration of CWEP-004 is being discussed. Version 4.0 We expect to accelerate development of CubicWeb 4, which exact roadmap is still to be discussed, but we may already want: be pyramid-based (remove twisted, auth management, etc.), do not have anything left of old dbapi and ClientConnection, integrate squareui as main (and only) web-ui "template" or remove web generation (almost) completely from cubicweb-core and provide it only through the cube system. Cubes cubicweb-bootstrap (0.6.5) cubicweb-brainomics (0.11.4, 0.12.0) cubicweb-clinipath (0.2.1, 0.2.2) cubicweb-collaboration (1.0.1) cubicweb-fastimport (0.2.0) cubicweb-file (1.16.1) cubicweb-forge (1.10.2) cubicweb-forgotpwd (0.6.0) cubicweb-invoice (0.8.0) cubicweb-leaflet (0.2.1, 0.3.0) cubicweb-mediaplayer (0.1.3, 0.1.4) cubicweb-medicalexp (0.12.3) cubicweb-osmfrance (0.2.2) cubicweb-person (1.8.1, 1.9.0) cubicweb-postgis (0.3.0) cubicweb-relationwidget (0.2.0) cubicweb-rqlcontroller (0.2.0, 0.3.0) cubicweb-squareui (0.3.7) cubicweb-subprocess (0.2.0) cubicweb-tracker (1.16.1) cubicweb-transactionlog (0.2.0, 0.2.1) cubicweb-worker (3.0.3, 3.0.4) cubicweb-wsme (0.1.0) cwclientlib (0.2.0) cwtags (1.1.0) New cubes and libraries cubicweb-i18nfield (0.1.0) cubicweb-pyramid (0.1.0) cubicweb-massmailing (0.1.0) cubicweb-wsme (0.1.0) pyramid-cubicweb (0.1.0, 0.1.1) rqlquery (0.1.0, 0.1.1) CWEPs Here is the status of open CubicWeb Evolution Proposals: to be written Work in progress Some work is in progress around CKAN, DCAT and othr Open Data and Semantic Web related technologies. Agenda Next roadmap meeting will be held at the beginning of january 2015 at Logilab, and Christophe and Dimitri (or Yann) are invited. Open Discussions Migration: AppObjects should not be loaded by default Have a look at Alembic the migration tool for SQLAlchemy and take inspiration from there [Less]

The datafeed API is one of the nice features of the CubicWeb framework. It makes it possible to easily build such things as a news aggregator (or even a semantic news feed reader), a LDAP importer or an application importing data from another web ... [More] platform. The underlying API is quite flexible and powerful. Yet, the documentation being quite thin, it may be hard to find one's way through. In this article, we'll describe the basics of the datafeed API and provide guiding examples. The datafeed API is essentially built around two things: a CWSource entity and a parser, which is a kind of AppObject. The CWSource entity defines a list of URL from which to fetch data to be imported in the current CubicWeb instance, it is linked to a parser through its __regid__. So something like the following should be enough to create a usable datafeed source [1]. create_entity('CWSource', name=u'some name', type='datafeed', parser=u'myparser') The parser is usually a subclass of DataFeedParser (from cubicweb.server.sources.datafeed). It should at least implement the two methods process and before_entity_copy. To make it easier, there are specialized parsers such as DataFeedXMLParser that already define process so that subclasses only have to implement the process_item method. Overview of the datafeed API Before going into further details about the actual implementation of a DataFeedParser, it's worth having in mind a few details about the datafeed parsing and import process. This involves various players from the CubicWeb server, namely: a DataFeedSource (from cubicweb.server.sources.datafeed), the Repository and the DataFeedParser. Everything starts from the Repository which loops over its sources and pulls data from each of these (this is done using a looping task which is setup upon repository startup). In the case of datafeed sources, Repository sources are instances of the aforementioned DataFeedSource class [2]. The DataFeedSource selects the appropriate parser from the registry and loops on each uri defined in the respective CWSource entity by calling the parser's process method with that uri as argument (methods pull_data and process_urls of DataFeedSource). If the result of the parsing step is successful, the DataFeedSource will call the parser's handle_deletion method, with the URI of the previously imported entities. Then, the import log is formatted and the transaction committed. The DataFeedSource and DataFeedParser are connected to an import_log which feeds the CubicWeb instance with a CWDataImport per data pull. This usually contains the number of created and updated entities along with any error/warning message logged by the parser. All this is visible in a table from the CWSource primary view. So now, you might wonder what actually happens during the parser's process method call. This method takes an URL from which to fetch data and processes further each piece of data (using a process_item method for instance). For each data-item: the repository is queried to retrieve or create an entity in the system source: this is done using the extid2entity method; this extid2entity method essentially needs two pieces of information:a so-called extid, which uniquely identifies an item in the distant source any other information needed to create or update the corresponding entity in the system source (this will be later refered to as the sourceparams) then, given the (new or existing) entity returned by extid2entity, the parser can perform further postprocessing (for instance, updating any relation on this entity). In step 1 above, the parser method extid2entity in turns calls the repository method extid2eid given the current source and the extid value. If an entry in the entities table matches with the specified extid, the corresponding eid (identifier in the system source) is returned. Otherwise, a new eid is created. It's worth noting that the created entity (in case the entity is to be created) is not complete with respect to the data model at this point. In order the entity to be completed, the source method before_entity_insertion is called. This is where the aforementioned sourceparams are used. More specifically, on the parser side the before_entity_copy method is called: it usually just updates (using entity.cw_set() for instance) the fetched entity with any relevant information. Case study: a news feeds parser Now we'll go through a concrete example to illustrate all those fairly abstract concepts and implement a datafeed parser which can be used to import news feeds. Our parser will create entities of type FeedArticle, which minimal data model would be: class FeedArticle(EntityType): title = String(fulltextindexed=True) uri = String(unique=True) author = String(fulltextindexed=True) content = RichString(fulltextindexed=True, default_format='text/html') Here we'll reuse the DataFeedXMLParser, not because we have XML data to parse, but because its interface fits well with our purpose, namely: it ships an item-based processing (a process_item method) and it relies on a parse method to fetch raw data. The underlying parsing of the news feed resources will be handled by feedparser. class FeedParser(DataFeedXMLParser): __regid__ = 'newsaggregator.feed-parser' The parse method is called by process, it should return a list tuples with items information. def parse(self, url): """Delegate to feedparser to retrieve feed items""" data = feedparser.parse(url) return zip(data.entries) Then the process_item method takes an individual item (i.e. an entry of the result obtained from feedparser in our case). It essentially defines an extid, here the uri of the feed entry (good candidate for unicity) and calls extid2entity with that extid, the entity type to be created / retrieved and any additional data useful for entity completion passed as keyword arguments. (The process_feed method call just transforms the results obtained from feedparser into a dict suitable for entity creation following the data model described above.) def process_item(self, entry): data = self.process_feed(entry) extid = data['uri'] entity = self.extid2entity(extid, 'FeedArticle', feeddata=data) The before_entity_copy method is called before the entity is actually created (or updated) in order to give the parser a chance to complete it with any other attribute that could be set from source data (namely feedparser data in our case). def before_entity_copy(self, entity, sourceparams): feeddata = sourceparams['feeddata'] entity.cw_edited.update(feeddata) And this is all what's essentially needed for a simple parser. Further details could be found in the news aggregator cube. More sophisticated parsers may use other concepts not described here, such as source mappings. Testing datafeed parsers Testing a datafeed parser often involves pulling data from the corresponding datafeed source. Here is a minimal test snippet that illustrates how to retrieve the datafeed source from a CWSource entity and to pull data from it. with self.admin_access.repo_cnx() as cnx: # Assuming one knows the URI of a CWSource. rset = cnx.execute('CWSource X WHERE X uri %s' % uri) # Retrieve the datafeed source instance. dfsource = self.repo.sources_by_eid[rset[0][0]] # Make sure it's parser matches the expected. self.assertEqual(dfsource.parser_id, '<my-parser-id>') # Pull data using an internal connection. with self.repo.internal_cnx() as icnx: stats = dfsource.pull_data(icnx, force=True, raise_on_error=True) icnx.commit() The resulting stats is a dictionnary containing eids of created and updated entities during the pull. In addition all entities created should have the cw_source relation set to the corresponding CWSource entity. Notes [1]It is possible to add some configuration to the CWSource entity in the form a string of configuration items (one per line). Noteworthy items are: the synchronization-interval; use-cwuri-as-url=no, which avoids using external URL inside the CubicWeb instance (leading to any link on an imported entity to point to the external source URI); delete-entities=[yes,no] which controls if entities not found anymore in the distant source should be deleted from the CubicWeb instance. [2]The mapping between CWSource entities' type (e.g. "datafeed") and DataFeedSource object is quite unusual as it does not rely on the vreg but uses a specific sources registry (defined in cubicweb.server.SOURCE_TYPES). [Less]