Welcome to the Warehouse!

Warehouse is the new codebase being developed to power the Python Packaging Index (PyPI). Python developers and users already know that PyPI is the official comprehensive repository of third-party open source Python packages (see Wikipedia). PyPI, maintained by the PSF, is where developers publish their software modules and from which package managers, such as pip, download packages. Given the important role played by PyPI, the Warehouse project is bound to have a huge impact on the continued use and growth of Python. 

While many volunteers have been working on the project, the biggest contributions have come from lead developer Donald Stufft and web-design specialist, Nicole Harris. Donald, based in the US, is a core contributor to PyPI (as well as to CPython, pip, virtualenv, Django & Cryptography), while UK-based Nicole runs a web development business, Kabu Creative.

The design goals, as stated on Nicole's website, were:

• To update the visual identity 


• To make packages more discoverable


• To accommodate the needs of both users and package maintainers 


• To give the project the same level of professionalism as a commercial project of the same scale 


• To ensure that the user experience reflects the Python community–a community that is welcoming, helpful and inclusive

Donald and Nicole have recently released a first look at the new design on the demo sites, Warehouse and Warehouse staging.

Looking great, thanks to Donald and Nicole's hard work, but there is plenty more to be done: writing code, writing the user guide (PyPUG), usability testing, and giving feedback. Details on how to volunteer and more info can be found on Nicole's site.

I would love to hear from readers. Please send feedback, comments, or blog ideas to me at msushi@gnosis.cx.

Read More

Twisted Trial Ported to Python 3!

Twisted, as many of you know, is an asynchronous, or event driven networking framework written in Python (https://twistedmatrix.com/trac/wiki). Twisted has been around for about a decade, offers many features, including low-level primitives and high-level interfaces, and works with many protocols (including HTTP, XMPP, NNTP, IMAP, SSH, IRC, FTP). 

Twisted Logo

Due to its maturity and complexity, Twisted requires a lot of time and effort to be completely ported to Python 3. Fortunately, the PSF was able to help fund some of this work; one recent result is the release of Twisted 15.4, which includes Twisted’s standard test-runner, Trial (codenamed "Trial by Fire").

The PSF Grant allowed core developer and Twisted release manager,

Amber (HawkOwl) Brown, to port Trial to Python 3. She recently sent the PSF this announcement:

“Just wanting to let you all know that a Twisted with the PSF-funded Trial Py3 port is now released. And a little example of it in action:

https://asciinema.org/a/cthr9xezlt8mxg5dp0n73fzc9. Again, many thanks for accepting the grant proposal – the ability to dedicate a significant chunk of time to this work has meant it was completed well sooner than if the grant had not been accepted.”

Due to certain differences between Python 3 and Python 2 (e.g., removal of ClassType and unbound methods), Amber tells us that the porting of Trial required a rewriting and retesting of the test suite loader. The work is mostly done and the current port duplicates most of Trial’s previous functionality with the exception of its distributed test runner (DistTrialRunner).

Specifically, the PSF grant allowed Amber to perform the following steps:

- Complete and test the Trial unittest loader 

- Fix the remaining failing Trial tests  

- Create a tool which runs only the portions of Twisted that have been ported to Python 3 for use in Twisted development  

- Break up the port into smaller pieces, put them up for review, and address the review comments  

- Merge the reviewed portions

Trial’s features–a front-end, the ability to handle Deferreds and asynchronous tests, and the capacity to build testcase-duration reactors, make testing much easier. The Twisted team will now be able to use Trial for continued Python 3 porting, while users of Twisted will be able to test their codebases more easily as they port them to Python 3. Because of Trial, we can look forward to Twisted 15.5 in the near future (and hope to see more users' code ported to Python 3, as well). As Amber tells us,

"15.5, coming soon, will come with another handful of ported modules, and the twistd application (a daemoniser + plugin runner, the recommended way of spawning long-running Twisted services)."

The PSF sends its gratitude and congratulations to Amber Brown and the Twisted team on this important accomplishment.

To learn more about Twisted, the following websites, video talks, and tutorials are available:

http://labs.twistedmatrix.com/

http://krondo.com/wp-content/uploads/2009/08/twisted-intro.html

http://www.ibm.com/developerworks/library/l-twist1/

http://pyvideo.org/video/2597/twisted-mixing

https://www.youtube.com/watch?v=_HZR7_ZBkYY

I would love to hear from readers. Please send feedback, comments, or blog ideas to me at msushi@gnosis.cx.

Read More

Read the Docs: growing with a little help from its friends at the PSF (and elsewhere)

Today's post, like the previous one, features a development project that the PSF has been delighted to fund once again this year.

On April 28, 2015, the PSF Board unanimously approved the following resolution:

RESOLVED, that the Python Software Foundation grant  $8,000 to Read the Docs, Inc. for developmental work.

What is RTD?

Looking for somewhere to host your open source project’s documentation in a way that will make it readily available, easy to find, fully searchable for your users, and exportable in PDF format, while at the same time offering you ease of use and the ability to add content as your project develops? Then, you’ll want to check out Read the Docs, the world’s largest documentation website for open source projects. 

Read the Docs

… hosts documentation, making it fully searchable and easy to find. You can import your docs using any major version control system, including Mercurial, Git, Subversion, and Bazaar. We support webhooks so your docs get built when you commit code. There’s also support for versioning so you can build docs from tags and branches of your code in your repository.

RTD’s History

RTD was created in 2010 by Eric Holscher, Charles Leifer, and Bobby Grace for the 2010 Django Dash. Eric tells the interesting story at Djangocon. A Django Dash is a coding contest that allows 48 hours for development and implementation of a project. Eric and his team considered what to do and decided that, since current documentation hosting was less than satisfactory, they could be of most help to the community by creating a web-based doc hosting solution. They agreed that Sphinx was the best document tool for Python, so they went with that.

According to Eric, 2011 was the year that saw RTD go … from a hobby project, into something projects depended on. At that point, they were hosting documentation for Celery, Fabric, Nose, py.test, Virtualenv, Pip, Django CMS, Django, Grapelli/Floppyforms/Sentry, mod_wsgi. Currently, they are hosting what Eric describes as a decent part of the Python ecosystem, including SQL Academy, Pyramid, Requests, Minecraft Overviewer, and many others. They have over 50 contributors, 7500 users, and get over 15,000,000 pageviews a month. The code for RTD is on GitHub and its documentation can be found on the site. Rackspace provides free hosting. A full list of features is available on the site.

Photo Credit: Aaron Hockley, October 2014 

Creative Commons license 2.00

Use of PSF Grant

The PSF award was part of a fundraising drive that opened at PyCon 2015 and brought in $24,000 USD from 157 contributions since then (see the RTD Blog). Corporate sponsors included Twilio, Sentry, DreamHost, and Lincoln Loop; with service sponsorships from Elastic Search, MaxCDN, and Gandi.

This funding will support RTD for 3 months of development work on the path toward sustainability as an open source project. More specifically, the funds will allow RTD to hire 2 part-time paid positions: Community Developer and Operations Developer (see RTD Blogpost for details and how to apply).

Furthermore, RTD intends to document its use of PSF grant money;  how development time is spent and how funds are allocated will be posted on RTD’s public Trello board.

If you’d like to help, you can contribute to RTD at Gratipay and you can follow them on Twitter.

I would love to hear from readers. Please send feedback, comments, or blog ideas to me at msushi@gnosis.cx.

Read More