pylint

Python3 The 2to3 script is a very useful tool. We can just use it to run over all code base, and end up with a python3 compatible code whilst keeping a python2 code base. To make our code python3 compatible, we do (or did) two things: small python2 ... [More] compatible modifications of our source code run 2to3 over our code base to generate a python3 compatible version However, we not only want to have one python3 compatible version, but also keep developping our software. Hence, we want to be able to easily test it for both python2 and python3. Furthermore if we use patches to get nice commits, this is starting to be quite messy. Let's consider this in the case of Pylint. Indeed, the workflow described before proved to be unsatisfying. I have two repositories, one for python2, one for python3. On the python3 side, I run 2to3 and store the modifications in a patch or a commit. Whenever I implement a fix or a functionality on either side, I have to test if it still works on the other side; but as the 2to3 modifications are often quite heavy, directly creating patches on one side and applying them on the other side won't work most of the time. Now say, I implement something in my python2 base and hold it in a patch or commit it. I can then pull it to my python3 repo: running 2to3 on all Pylint is quite slow: around 30 sec for Pylint without the tests, and around 2 min with the tests. (I'd rather not imagine how long it would take for say CubicWeb). even if I have all my 2to3 modifications on a patch, it takes 5-6 sec to "qpush" or "qpop" them all. Commiting the 2to3 changes instead and using: hg pull -u --rebase is not much faster. If I don't use --rebase, I will have merges on each pull up. Furthermore, we often have either a patch application failure, merge conflict or end up with something which is not python3 compatible (like a newly introduced "except Error, exc"). So quite often, I will have to fix it with: hg revert -r REV <broken_files> 2to3 -nw <broken_files> hg qref # or hg resolve -m; hg rebase -c Suppose that 2to3 transition worked fine, or that we fixed it. I run my tests with python3 and see it does not work; so I modify the patch: it all starts again; and the new patch or the patch modification will create a new head in my python3 repo... 2to3 Fixers Considering all that, let's investigate 2to3: it comes with a lot of fixers that can be activated or desactived. Now, a lot of them fix just very seldom use cases or stuff deprecated since years. On the other hand, the 2to3 fixers work with regular expressions, so the more we remove, the faster 2to3 should be. Depending on the project, most cases will just not appear, and for the others, we should be able to find other means of disabling them. The lists proposed here after are just suggestions, it will depend on the source base and other overall considerations which and how fixers could actually be disabled. python2 compatible Following fixers are 2.x compatible and should be run once and for all (and can then be disabled on daily conversion usage): apply execfile (?) exitfunc getcwdu has_key idioms ne nonzero paren repr standarderror sys_exec tuple_params ws_comma compat This can be fixed using imports from a "compat" module like the logilab.common.compat module which holds convenient compatible objects. callable exec filter (Wraps filter() usage in a list call) input intern itertools_imports itertools map (Wraps map() in a list call) raw_input reduce zip (Wraps zip() usage in a list call) strings and bytes Maybe they could also be handled by compat: basestring unicode print For print for example, we could think of a once-and-for-all custom fixer, that would replace it by a convenient echo function (or whatever name you like) defined in compat. manually Following issues could probably be fixed manually: dict (it fixes dict iterator methods; it should be possible to have code where we can disable this fixer) import (Detects sibling imports; we could convert them to absolute import) imports, imports2 (renamed modules) necessary These changes seem to be necessary: except long funcattrs future isinstance (Fixes duplicate types in the second argument of isinstance(). For example, isinstance(x, (int, int)) is converted to isinstance(x, (int))) metaclass methodattrs numliterals next raise Consider however that a lot of them might never be used in some projects, like long, funcattrs, methodattrs and numliterals or even metaclass. Also, isinstance is probably motivated by long to int and unicode to str conversions and hence might also be somehow avoided. don't know Can we fix these one also with compat ? renames throw types urllib xrange xreadlines 2to3 and Pylint Pylint is a special case since its test suite has a lot of bad and deprecated code which should stay there. However, in order to have a reasonable work flow, it seems that something must be done to reduce the 1:30 minutes of 2to3 parsing of the tests. Probably nothing could be gained from the above considerations since most cases just should be in the tests, and actually are. Realise that We can expect to be supporting python2 and python3 for several years in parallel. After a quick look, we see that 90 % of the refactorings of test/input files are just concerning the print statements; more over most of them have nothing to do with the tested functionality. Hence a solution might be to avoid to run 2to3 on the test/input directory, since we already have a mechanism to select depending on python version whether a test file should be tested or not. To some extend, astng is a similar case, but the test suite and the whole project is much smaller. [Less]

First of all, I've to say that pylint bugs day wasn't that successful in term of 'community event': I've been sprinting almost alone. My Logilab's felows were tied to customer projects, and no outside people shown up on jabber. Fortunatly Tarek Ziade ... [More] came to visit us, and that was a nice opportunity to talk about pylint, distribute, etc ... Thank you Tarek, you saved my day ;) As I felt a bit alone, I decided to work on somethings funnier than bug fixing: refactoring! First, I've greatly simplified the command line: enable-msg/enable-msg-cat/enable-checker/enable-report and their disable-* counterparts were all merged into single --enable/--disable options. I've also simplified "pylint --help" output, providing a --long-help option to get what we had before. Generic support in `logilab.common.configuration of course. And last but not least, I refactored pylint so we can have multiple checkers with the same name. The idea behind this is that we can split checker into smaller chunks, basically only responsible for one or a few related messages. When pylint runs, it only uses necessary checkers according to activated messages and reports. When all checkers will be splitted, it should improve performance of "pylint --error-only". So, I can say I'm finally happy with the results of that pylint bugs day! And hopefuly we will be more people for the next edition... [Less]

We are happy to announce the Astng 0.20.0 and Pylint 0.20.0 releases. Pylint is a static code checker based on Astng, both depending on logilab-common 0.49. Astng Astng 0.20.0 is a major refactoring: instead of parsing and modifying the syntax tree ... [More] generated from python's _ast or compiler.ast modules, the syntax tree is rebuilt. Thus the code becomes much clearer, and all monkey patching will eventually disappear from this module. Speed improvement is achieved by caching the parsed modules earlier to avoid double parsing, and avoiding some repeated inferences, all along fixing a lot of important bugs. Pylint Pylint 0.20.0 uses the new Astng, and fixes a lot of bugs too, adding some new functionality: parameters with leading "_" shouldn't count as "local" variables warn on assert( a, b ) warning if return or break inside a finally specific message for NotImplemented exception We would like to thank Chmouel Boudjnah, Johnson Fletcher, Daniel Harding, Jonathan Hartley, Colin Moris, Winfried Plapper, Edward K. Ream and Pierre Rouleau for their contributions, and all other people helping the project to progress. [Less]

Hey guys, we'll hold the next pylint bugs day on april 16th 2010 (friday). If some of you want to come and work with us in our Paris office, you'll be much welcome. Else you can still join us on jabber / irc: jabber: chat room [email protected] irc: #public on irc://irc.logilab.org See you then!

The first pylint bug day held on wednesday 25th. Four members of the Logilab crew and two other people spent the day working on pylint. Several patches submitted before the bug day were processed and some tickets were closed. Charles Hébert added ... [More] James Lingard's patches for string formatting and is working on several improvements. Vincent Férotin submitted a patch for simple messages listings. Sylvain Thenault fixed significant inference bugs in astng (an underlying module of pylint managing the syntax tree). Émile Anclin began a major astng refactoring to take advantage of new python2.6 functionality. For my part, I made several improvements to the test suite. I applied James Lingard patches for ++ operator and generalised it to -- too. I also added a new checker for function call arguments submitted by James Lingard once again. Finally I improved the message filtering of the --errors-only options. We thank Maarten ter Huurne, Vincent Férotin for their participation and of course James Lingard for submitting numerous patches. Another pylint bug day will be held in a few months. image under creative commons by smccann [Less]

Remember that the first pylint bug day will be held on wednesday, november 25, from around 8am to 8pm in the Paris (France) time zone. We'll be a few people at Logilab and hopefuly a lot of other guys all around the world, trying to make pylint ... [More] better. Join us on the #public conference room of conference.jabber.logilab.org, or if you prefer using an IRC client, join #public on irc.logilab.org which is a gateway to the jabber forum. And if you're in Paris, come to work with us in our office. People willing to help but without knowledge of pylint internals are welcome, it's the perfect occasion to learn a lot about it, and to be able to hack on pylint in the future! [Less]

Since we don't stop being overloaded here at Logilab, and we've got some encouraging feedback after the "Pylint needs you" post, we decided to take some time to introduce more "community" in pylint. And the easiest thing to do, rather sooner than ... [More] later, is a irc/jabber synchronized bug day, which will be held on Wednesday november 25. We're based in France, so main developpers will be there between around 8am and 19pm UTC+1. If a few of you guys are around Paris at this time and wish to come at Logilab to sprint with us, contact us and we'll try to make this possible. The focus for this bug killing day could be: using logilab.org tracker : getting an account, submitting tickets, triaging existing tickets... using mercurial to develop pylint / astng guide people in the code so they're able to fix simple bugs We will of course also try to kill a hella-lotta bugs, but the main idea is to help whoever wants to contribute to pylint... and plan for the next bug-killing day ! As we are in the process of moving to another place, we can't organize a sprint yet, but we should have some room available for the next time, so stay tuned :) [Less]

Après plusieurs mois au point mort ou presque, Sylvain a pu hier soir publier des versions corrigeant un certain nombre de bogues dans pylint et astng ([1] et [2]). Il n'en demeure pas moins qu'à Logilab, nous manquons de temps pour faire baisser la ... [More] pile de tickets ouverts dans le tracker de pylint. Si vous jetez un œuil dans l'onglet Tickets, vous y trouverez un grand nombre de bogues en souffrance et de fonctionalités indispensables (certaines peut-être un peu moins que d'autres...) Il est déjà possible de contribuer en utilisant mercurial pour fournir des patches, ou en signalant des bogues (aaaaaaaaaarg ! encore des tickets !) et certains s'y sont mis, qu'ils en soient remerciés. Maintenant, nous nous demandions ce que nous pourrions faire pour faire avance Pylint, et nos premières idées sont : organiser un petit sprint de 3 jours environ organiser des jours de "tuage de ticket", comme ça se pratique sur différents projets OSS Mais pour que ça soit utile, nous avons besoin de votre aide. Voici donc quelques questions : est-ce que vous participeriez à un sprint à Logilab (à Paris, France), ce qui nous permettrait de nous rencontrer, de vous apprendre plein de choses sur le fonctionnement de Pylint et de travailler ensemble sur des tickets qui vous aideraient dans votre travail ? si la France c'est trop loin, où est-ce que ça vous arrangerait ? seriez-vous prêt à vous joindre à nous sur le serveur jabber de Logilab ou sur IRC, pour participer à une chasse au ticket (à une date à déterminer). Si oui, quel est votre degré de connaissance du fonctionnement interne de Pylint et astng ? Vous pouvez répondre en commentant sur ce blog (pensez à vous enregistrer en utilisant le lien en haut à droite sur cette page) ou en écrivant à [email protected]. Si nous avons suffisamment de réponses positives nous organiserons quelque chose. [Less]

After several months with no time to fix/enhance pylint beside answering email and filing tickets, I've finally tackled some tasks yesterday night to publish bug fixes releases ([1] and [2]). The problem is that we don't have enough free time at ... [More] Logilab to lower the number of tickets in pylint tracker page . If you take a look at the ticket tab, you'll see a lot of pendings bug and must-have features (well, and some other less necessary...). You can already easily contribute thanks to the great mercurial dvcs, and some of you do, either by providing patches or by reporting bugs (more tickets, iiirk ! ;) Thank you all btw !! Now I was wondering what could be done to make pylint going further, and the first ideas which came to my mind was : do ~3 days sprint do some 'tickets killing' days, as done in some popular oss projects But for this to be useful, we need your support, so here are some questions for you: would you come to a sprint at Logilab (in Paris, France), so you can meet us, learn a lot about pylint, and work on tickets you wish to have in pylint? if France is too far away for most people, would you have another location to propose? would you be on jabber for a tickets killing day, providing it's ok with your agenda? if so, what's your knowledge of pylint/astng internals? you may answer by adding a comment to this blog (please register first by using the link at the top right of this page) or by mail to [email protected]. If we've enough positive answers, we'll take the time to organize such a thing. [Less]

I'm pleased to announce releases of pylint 0.18, logilab-astng 0.19 and logilab-common 0.39. All these packages should now be cleanly available through easy install. Also, happy pylint users will get: fixed python 2.6 support (pylint/astng tested ... [More] from 2.4 to 2.6) get source code (and so astng) for zip/egg imports some understanding of the property decorator and of unbound methods some false positives fixed and others minor improvments See projects home page and ChangeLog for more information: http://www.logilab.org/project/pylint http://www.logilab.org/project/logilab-astng http://www.logilab.org/project/logilab-common Please report any problem / question to the [email protected] mailing-list. Enjoy! [Less]