A Guide For Filesystem Implementers¶
PyFilesystem objects are designed to be as generic as possible and still expose the full filesystem functionality. With a little care, you can write a wrapper for your filesystem that allows it to work interchangeably with any of the built-in FS classes and tools.
To create a working PyFilesystem interface, derive a class from
FS and implement the 9 Essential Methods.
The base class uses these essential methods as a starting point for providing a lot of extra functionality,
but in some cases the default implementation may not be the most efficient.
For example, most filesystems have an atomic way of moving a file from one location to another without having to copy data,
whereas the default implementation of
move() method must copy all the bytes in the source file to the destination file.
Any of the Non - Essential Methods may be overridden, but efficient custom versions of the following methods will have the greatest impact on performance:
For network based filesystems (i.e. where the physical data is pulled over a network), there are a few methods which can reduce the number of round trips to the server, if an efficient implementation is provided:
The generator methods (beginning with
i) are intended for use with filesystems that contain a lot of files,
where reading the directory in one go may be expensive.
Other methods in the Filesystem Interface are unlikely to require a non-default implementation, but there is nothing preventing you from implementing them – just be careful to use the same signature and replicate expected functionality.
With the exception of the constructor, FS methods should throw
FSError exceptions in preference to any implementation-specific exception classes,
so that generic exception handling can be written.
The constructor may throw a non-FSError exception, if no appropriate FSError exists.
The rationale for this is that creating an FS interface may require specific knowledge,
but this shouldn’t prevent it from working with more generic code.
If specific exceptions need to be translated in to an equivalent FSError, pass the original exception class to the FSError constructor with the ‘details’ keyword argument.
For example, the following translates some fictitious exception in to an FSError exception, and passes the original exception as an argument.:
try: someapi.open(path, mode) except someapi.UnableToOpen, e: raise errors.ResourceNotFoundError(path=path, details=e)
Any code written to catch the generic error, can also retrieve the original exception if it contains additional information.
All PyFilesystem methods, other than the constructor, should be thread-safe where-ever possible.
One way to do this is to pass
threads_synchronize=True to the base constructor and use the
synchronize() decorator to lock the FS object when a method is called.
If the implementation cannot be made thread-safe for technical reasons, ensure that
getmeta() method is designed to return implementation specific information.
PyFilesystem implementations should return as much of the standard set of meta values as possible.
Implementations are also free to reserve a dotted namespace notation for themselves, to provide an interface to highly specific information.
If you do this, please avoid generic terms as they may conflict with existing or future implementations.
"bobs_ftpfs.author", rather than
If your meta values are static, i.e. they never change, then create a dictionary class attribute called
_meta in your implementation that contains all the meta keys and values.
getmeta implementation will pull the meta values from this dictionary.
The following methods are required for a minimal Filesystem interface:
open()Opens a file for read/writing
isfile()Check whether the path exists and is a file
isdir()Check whether a path exists and is a directory
listdir()List the contents of a directory
makedir()Create a new directory
remove()Remove an existing file
removedir()Remove an existing directory
rename()Atomically rename a file or directory
getinfo()Return information about the path e.g. size, mtime
Non - Essential Methods¶
The following methods have default implementations in
FS and aren’t required for a functional FS interface. They may be overridden if an alternative implementation can be supplied:
copy()Copy a file to a new location
copydir()Recursively copy a directory to a new location
desc()Return a short descriptive text regarding a path
exists()Check whether a path exists as file or directory
listdirinfo()Get a directory listing along with the info dict for each entry
ilistdir()Generator version of the listdir method
ilistdirinfo()Generator version of the listdirinfo method
getpathurl()Get an external URL at which the given file can be accessed, if possible
getsyspath()Get a file’s name in the local filesystem, if possible
getmeta()Get the value of a filesystem meta value, if it exists
getmmap()Gets an mmap object for the given resource, if supported
hassyspath()Check if a path maps to a system path (recognized by the OS)
haspathurl()Check if a path maps to an external URL
hasmeta()Check if a filesystem meta value exists
move()Move a file to a new location
movedir()Recursively move a directory to a new location
settimes()Sets the accessed and modified times of a path