Python Posts

166 posts tagged with python

I've just released version 2.0.3 of PyFilesystem.

PyFilesystem is an abstraction layer for filesystems (or anything that resembles a filesystem). See this post for more details.

New in this version is a TarFS filesystem contributed by Martin Larralde, which compliments ZipFS nicely.

Contributed by gpcimino, the copy module was extended new functionality to selectively copy only new files from one filesystem to another. Very useful for backups.

This is in addition to bugfixes, and improved documentation.

I recently I had the opportunity to speed up some Websocket code that was a major bottleneck. The final solution was 60X (!) faster than the first pass, and an interesting exercise in optimizing inner loops.

When you send data via a websocket it is masked with a randomly generated four byte key. This masking provides protection from certain exploits against proxies (details here). The algorithm is super simple; essentially each byte in the payload is XORed with a corresponding byte from the key.

The first version of my XOR mask function looked this:

That's the kind of unfussy and elegant code you want to write for a job interview. If that's the solution that first came to mind, then you know your Python. Alas, this code is as slow as it is beautiful. continue reading…

I like to imagine components of a Python web framework as personalities. Here's how I would imagine a conversation between a typical view and template would go down:

View Here's that data I just looked up.

Template Don't like it.

View What's wrong with it?

Template Format's wrong.

View You can change the format if you like.

Template Don't wanna. You change it.

View OK. Fine. I'll change it for you.

Which is why I wanted Moya templates to be very capable in processing data, but still keep them designer friendly. If the template data is sane then the template should be able to deal with it, without going back to the back-end developer. continue reading…

I'd like to announce version 2.0.0 of PyFilesystem, which is now available on PyPi.

PyFilesystem is a Python module I started some time in 2008, and since then it has been very much a part of my personal standard library. I've used it in personal and professional projects, as have many other developers and organisations.

If you aren't familiar with PyFilesystem; it's an abstraction layer for filesystems. Essentially anything with files and directories (hard-drive, zip file, ftp server, network filesystems etc.) may be wrapped with a common interface. With it, you can write code that is agnostic as to where the files are physically located.

Here's a quick example that recursively counts the lines of code in a directory: continue reading…

It's not often I come across something in Python that surprises me. Especially in something as mundane as string operations, but I guess Python still has a trick or two up its sleeve.

Have a look at this string:

How many possible sub-strings are in s? To put it another away, how many values of x are there where the expression x in s is true?

Turns out it is 2.

2?

Yes, 2.

The empty string is in the string "A". In fact, it's in all the strings.

Turns out the empty string has been hiding every where in my code.

Not a complaint, I'm sure the rationale is completely sound. And it turned out to be quite useful. I had a couple of lines of code that looked something like this: continue reading…

This SO post prompted me to think about how you would go about customizing a entire Django site (not just an app). Particularly if it is a third party project, still in development.

Forking the site repository is a solution, but the more customizations you make, the more awkward it becomes to merge upstream fixes and features. The problem is compounded if you have multiple deploys, with custom templates, assets, and settings.

I wanted to do a quick write-up of how this would be done in Moya, where customization is more of a first-class concept.

I'll briefly go over how you would serve a Moya site with a custom template, without modifying any of the original files. The project I'll be customizing is Moya Techblog which power this blog. continue reading…

Inthing can take a screenshot of your desktop then post it online.

Here's a quick example:

Alternatively you can do the same thing from the command line with the inthing app. The following is equivalent to the Python code (assuming you have set up the stream and password environment variables):

See the docs for more details.

Screenshot capabilities courtesy of the excellent pyscreenshot library.

Screenshot taken with Stream.screenshot

Just landed in inthing is a new and quite interesting feature.

Version 0.1.4 adds a capture method which will record all standard output, i.e. anything you print to the terminal. It works as a context manager. Here's an example:

Any print statement inside the with block will be captured and posted online with the block exits.

You can also do something similar from the command line, with the inthing capture subcommand, which posts anything you pipe in to it as an event.

lets say you wanted to post the version of all you installed Python packages online. You could do something like the following:

For more information see the Inthing docs.

Inthing is still technically in beta, but these features are quite solid. Please give them a try, and let me know how it goes!

The Django code in this post comes from the official Django tutorial.

I've tried not to be disingenuous with the comparison, and I'm only going to compare like with like, so I can show code from both frameworks and let you draw your own conclusions. I'll cover the areas where they differ in another post.

Both Moya and Django use models to map databases on to familiar data structures. In the case of Moya, the mapping is done with SQLAlchemy. Django uses its own ORM.

Here's the models.py from the Django tutorial and a Moya version: continue reading…

One of my goals for inthing.io was to make posting events realtime, in that events appear without a page refresh, and within a fraction of a second. And that largely seems to work.

Here's a quick screencast that shows it working:

I'll post about how it works in detail at some point, but the general gist is that there is a Tornado websocket server that inthing uses to broadcast information about updates. That server may be worth open-sourcing if there is enough interest. It could be useful for other projects, and its entirely independent.