Pyfilesystem Posts

I was reading a post by Trey Hunner on why pathlib is great, where he makes the case that pathlib is a better choice than the standard library alternatives that preceded it. I wouldn't actually disagree with a word of it. He's entirely correct. You should probably be using pathlib were it fits.

Personally, however, I rarely use pathlib, because I find that for the most part, PyFilesystem is a better choice. I'd like to take some of the code examples from Trey's post and re-write them using PyFilesystem, just so we can compare.

The first example from Trey's post, creates a folder then moves a file into it. Here it is:

The code above is straightforward, and hides the gory platform details which is a major benefit of pathlib over os.path. continue reading…

I recently added a number of examples on working with files and directories with Python and PyFilesystem.

The first example, is count_py.py which recursively sums up the number of bytes stored in a directory, and renders it in a "friendly" format.

This script takes a path or a URL. For instance python3 count_py.py ~/projects will sum up the bytes of Python in your projects folder.

For me, this gives the following output:

This script accepts a path, but it will also work with any supported filesystem. So you could sum the bytes of Python in an archive or cloud server. continue reading…

I've released version 2.0.4 of PyFilesystem.

PyFilesystem is an abstraction layer for filesystems (or anything that resembles a filesystem). See this post for more details.

This version adds support for Python3.6 os.pathlike to the OSFS class. This is in oft-requested feature, but it only affects the OSFS constructor; since FS methods use a standardized path format.

Also in the version is a contribution by Martin Larralde which implements an elegant extension mechanism. Essentially, if you layout your filesystem implementation according to the convention, then PyFilesystem will be aware of it automatically. So open_fs('awesomefs://foo/bar') would work without explicitly adding awesomefs to the core library. continue reading…

I've just released version 2.0.3 of PyFilesystem.

PyFilesystem is an abstraction layer for filesystems (or anything that resembles a filesystem). See this post for more details.

New in this version is a TarFS filesystem contributed by Martin Larralde, which compliments ZipFS nicely.

Contributed by gpcimino, the copy module was extended new functionality to selectively copy only new files from one filesystem to another. Very useful for backups.

This is in addition to bugfixes, and improved documentation.

Ben Timby has committed code to PyFilesystem that lets you expose any filesystem over FTP. We've had the ability to serve filesystems over SFTP (secure ftp) and XMLRPC for a while, but plain old FTP was a glaring omission–until now.

You can serve the current directory programatically with something like the following:

The same functionality is also available to the fsserve command. The following is equivalent to the above code, but from the command line:

You'll probably need root privileges (i.e. sudo) on Linux for these examples.

With the server running, you can browse the files on your home directory with an ftp client, or by typing “ftp://127.0.0.1” in to your browser. Any of the other supported filesystems can be served in the same way. continue reading…

If you are writing an application of any size, it will most likely require a number of files to run – files which could be stored in a variety of possible locations. Furthermore, you will probably want to be able to change the location of those files when debugging and testing. You may even want to store those files somewhere other than the user's hard drive.

Any engineer worth his salt will recognise that the file locations should be stored in some kind of configuration file and the code to read the files in question should be factored out so that it isn't just scattered at points where data is read or written. In this post I'll present a way of doing just that by creating a virtual filesystem with PyFilesystem.

You'll need the most recent version of PyFilesystem from SVN to run this code. continue reading…

There have been some pretty exciting developments in PyFilesystem since version 0.3 was released – Ryan Kelly and myself have been hard at work, and there have been a number of excellent contributions to the code base from other developers. Version 0.4 will be released some time in January, but I'd like to give you a preview of some new features before the next version lands.

Pyfilesystem is a Python module that provides a simplified common interface to many types of filesystem.

It is now possible to open any of the supported filesystems from a URL in this format, which makes it very easy to specify a filesystem (or individual file) from the command line or a config file. Here's a quick example that opens a bunch of quite different filesystems: continue reading…