PyFilesystem 0.3 released
I am pleased to announce a new version of PyFilesystem (0.3), which is a Python module that provides a common interface to many kinds of filesystem. Basically it provides a way of working with files and directories that is exactly the same, regardless of how and where the file information is stored. Even if you don't plan on working with anything other than the files and directories on your hard-drive, PyFilesystem can simplify your code and reduce the potential of error.
PyFilesystem is a joint effort by myself and Ryan Kelly, who has created a number of new FS implementations such as Amazon S3 support and Secure FTP, and some pretty cool features such as FUSE support and Django storage integration.
As an example of how awesome this package is, take a look at the following 6 lines of code, which creates a ramdrive:
from fs.osfs import OSFS from fs.memoryfs import MemoryFS from fs.expose import fuse home_fs = OSFS('~/') home_fs.makedir('ramdrive', allow_recreate=True) fuse.mount(MemoryFS(), home_fs.getsyspath('ramdrive'))
If you run this, a directory called ramdrive
will appear in your home folder, the contents of which are stored purely in memory.
I prepared a screencast that gives a quick demonstration of some features – because if a picture is worth a thousand words, this video must be worth fifteen thousand words a second:
PyFilesystem screencast from Will McGugan on Vimeo.
See the project page on google code for more information, including API docs. There are also a couple of blog posts that will give a some more context.
This release has reached a good level of stability and maturity. I'd like to invite as many Pythonistas as possible to check out this module and possibly contribute to the project.
This is the first time I stumble upon this library. I am particularly interested by the S3FS.
It seems to me particularly interesting to archive files and directories to S3. Do you have any idea on how you would mirror a large directory structure ? By large I mean 20 Gb (larger than the ram)
indeed my question is does “fs.utils.copydir” is limited by the ram or not ?
Thank you
http://eagain.net/talks/pythonic-fs-2009/
Because that is a really neat tool I've used a few times.
Koen, read-only filesystems are supported. The implementation just needs to prohibit opening files in write mode and any operations which alter the directory structure.
It would make things faster too I suspect.
http://code.google.com/intl/es/apis/storage/
sudo gpasswd -a YOUR_USER_NAME fuse
The documentation for the lower-level tinkering is more than enough to get me started on that part, but fs.expose.django_storage usage documentation is a bit sparse (ok, nonexistant ;-). Is there an example (good or otherwise) of using PyFilesystem with Django that someone could point me to?
Thanks for making my life a little easier.
Joe
If you have any problems, try the discussion group:
http://groups.google.com/group/pyfilesystem-discussion
This projects rocks out loud!
I ever wanted a FS abstraction in python, it is REALLY useful for so many things.
One example, when using test-driven development, it really sucks to monkey patch os and shutil.
It can look silly, but is a awesome usage for a FS abstraction layer …
Thanks a lot!
Are there any chances to run/execute files stored in memoryfs? Would it be technically possible?
Will
I have tested your ramdrive example with exposing fs.memoryfs to the os by fs.expose.fuse but it is impossible to set x permission to files under mount dir from os side.
What fuse alternative do you mean?
Can I ask what books or methodologies you use to design software?
Thanks,
Chris
Great lib, thank you guys & happy new year :)
Just released a django+ExtJS explorer with pyfilesystem backend and drag&drop UI. https://github.com/revolunet/django-extjs-filebrowser
Works like a charm.
I miss you on github.