fs.base

This module contains the basic FS interface and a number of other essential interfaces.

fs.base.FS

All Filesystem objects inherit from this class.

class fs.base.FS(thread_synchronize=True)

The base class for Filesystem abstraction objects. An instance of a class derived from FS is an abstraction on some kind of filesystem, such as the OS filesystem or a zip file.

The base class for Filesystem objects.

Parameters:thread_synconize – If True, a lock object will be created for the object, otherwise a dummy lock will be used.
browse(hide_dotfiles=False)

Displays the FS tree in a graphical window (requires wxPython)

Parameters:hide_dotfiles – If True, files and folders that begin with a dot will be hidden
cache_hint(enabled)
Recommends the use of caching. Implementations are free to use or
ignore this value.
Parameters:enabled – If True the implementation is permitted to aggressively cache directory structure / file information. Caching such information can speed up many operations, particularly for network based filesystems. The downside of caching is that changes made to directories or files outside of this interface may not be picked up immediately.
cachehint(enabled)
Recommends the use of caching. Implementations are free to use or
ignore this value.
Parameters:enabled – If True the implementation is permitted to aggressively cache directory structure / file information. Caching such information can speed up many operations, particularly for network based filesystems. The downside of caching is that changes made to directories or files outside of this interface may not be picked up immediately.
close()

Close the filesystem. This will perform any shutdown related operations required. This method will be called automatically when the filesystem object is garbage collected, but it is good practice to call it explicitly so that any attached resourced are freed when they are no longer required.

copy(src, dst, overwrite=False, chunk_size=65536)

Copies a file from src to dst.

Parameters:
  • src (string) – the source path
  • dst (string) – the destination path
  • overwrite (bool) – if True, then an existing file at the destination may be overwritten; If False then DestinationExistsError will be raised.
  • chunk_size (bool) – size of chunks to use if a simple copy is required (defaults to 64K).
copydir(src, dst, overwrite=False, ignore_errors=False, chunk_size=16384)

copies a directory from one location to another.

Parameters:
  • src (string) – source directory path
  • dst (string) – destination directory path
  • overwrite (bool) – if True then any existing files in the destination directory will be overwritten
  • ignore_errors (bool) – if True, exceptions when copying will be ignored
  • chunk_size – size of chunks to use when copying, if a simple copy is required (defaults to 16K)
createfile(path, wipe=False)

Creates an empty file if it doesn’t exist

Parameters:
  • path – path to the file to create
  • wipe – if True, the contents of the file will be erased
desc(path)

Returns short descriptive text regarding a path. Intended mainly as a debugging aid.

Parameters:path – A path to describe
Return type:str
exists(path)

Check if a path references a valid resource.

Parameters:path (string) – A path in the filesystem
Return type:bool
getcontents(path, mode='rb', encoding=None, errors=None, newline=None)

Returns the contents of a file as a string.

Parameters:
  • path – A path of file to read
  • mode – Mode to open file with (should be ‘rb’ for binary or ‘t’ for text)
  • encoding – Encoding to use when reading contents in text mode
  • errors – Unicode errors parameter if text mode is use
  • newline – Newlines parameter for text mode decoding
Return type:

str
Returns:

file contents
getinfo(path)

Returns information for a path as a dictionary. The exact content of this dictionary will vary depending on the implementation, but will likely include a few common values. The following values will be found in info dictionaries for most implementations:

  • “size” - Number of bytes used to store the file or directory
  • “created_time” - A datetime object containing the time the resource was created
  • “accessed_time” - A datetime object containing the time the resource was last accessed
  • “modified_time” - A datetime object containing the time the resource was modified
Parameters:

path (string) – a path to retrieve information for
Return type:

dict
Raises:
getinfokeys(path, *keys)

Get specified keys from info dict, as returned from getinfo. The returned dictionary may not contain all the keys that were asked for, if they aren’t available.

This method allows a filesystem to potentially provide a faster way of retrieving these info values if you are only interested in a subset of them.

Parameters:
  • path – a path to retrieve information for
  • keys – the info keys you would like to retrieve
Return type:

dict
getmeta(meta_name, default=<class 'fs.base.NoDefaultMeta'>)

Retrieve a meta value associated with an FS object.

Meta values are a way for an FS implementation to report potentially useful information associated with the file system.

A meta key is a lower case string with no spaces. Meta keys may also be grouped in namespaces in a dotted notation, e.g. ‘atomic.namespaces’. FS implementations aren’t obliged to return any meta values, but the following are common:

  • read_only True if the file system cannot be modified
  • thread_safe True if the implementation is thread safe
  • network True if the file system requires network access
  • unicode_paths True if the file system supports unicode paths
  • case_insensitive_paths True if the file system ignores the case of paths
  • atomic.makedir True if making a directory is an atomic operation
  • atomic.rename True if rename is an atomic operation, (and not implemented as a copy followed by a delete)
  • atomic.setcontents True if the implementation supports setting the contents of a file as an atomic operation (without opening a file)
  • free_space The free space (in bytes) available on the file system
  • total_space The total space (in bytes) available on the file system
  • virtual True if the filesystem defers to other filesystems
  • invalid_path_chars A string containing characters that may not be used in paths

FS implementations may expose non-generic meta data through a self-named namespace. e.g. "somefs.some_meta"

Since no meta value is guaranteed to exist, it is advisable to always supply a default value to getmeta.

Parameters:
  • meta_name – The name of the meta value to retrieve
  • default – An option default to return, if the meta value isn’t present
Raises:

fs.errors.NoMetaError – If specified meta value is not present, and there is no default
getmmap(path, read_only=False, copy=False)

Returns a mmap object for this path.

See http://docs.python.org/library/mmap.html for more details on the mmap module.

Parameters:
  • path – A path on this filesystem
  • read_only – If True, the mmap may not be modified
  • copy – If False then changes wont be written back to the file
Raises:

fs.errors.NoMMapError – Only paths that have a syspath can be opened as a mmap
getpathurl(path, allow_none=False)

Returns a url that corresponds to the given path, if one exists.

If the path does not have an equivalent URL form (and allow_none is False) then a NoPathURLError exception is thrown. Otherwise the URL will be returns as an unicode string.

Parameters:
  • path – a path within the filesystem
  • allow_none (bool) – if true, this method can return None if there is no URL form of the given path
Raises:

fs.errors.NoPathURLError – If no URL form exists, and allow_none is False (the default)
Return type:

unicode
getsize(path)

Returns the size (in bytes) of a resource.

Parameters:path (string) – a path to the resource
Returns:the size of the file
Return type:integer
getsyspath(path, allow_none=False)

Returns the system path (a path recognized by the OS) if one is present.

If the path does not map to a system path (and allow_none is False) then a NoSysPathError exception is thrown. Otherwise, the system path will be returned as a unicode string.

Parameters:
  • path – a path within the filesystem
  • allow_none (bool) – if True, this method will return None when there is no system path, rather than raising NoSysPathError
Raises:

fs.errors.NoSysPathError – if the path does not map on to a system path, and allow_none is set to False (default)
Return type:

unicode
hasmeta(meta_name)

Check that a meta value is supported

Parameters:meta_name – The name of a meta value to check
Return type:bool
haspathurl(path)

Check if the path has an equivalent URL form

Parameters:path – path to check
Returns:True if path has a URL form
Return type:bool
hassyspath(path)

Check if the path maps to a system path (a path recognized by the OS).

Parameters:path – path to check
Returns:True if path maps to a system path
Return type:bool
ilistdir(path='./', wildcard=None, full=False, absolute=False, dirs_only=False, files_only=False)

Generator yielding the files and directories under a given path.

This method behaves identically to listdir() but returns an generator instead of a list. Depending on the filesystem this may be more efficient than calling listdir() and iterating over the resulting list.

ilistdirinfo(path='./', wildcard=None, full=False, absolute=False, dirs_only=False, files_only=False)

Generator yielding paths and path info under a given path.

This method behaves identically to listdirinfo() but returns an generator instead of a list. Depending on the filesystem this may be more efficient than calling listdirinfo() and iterating over the resulting list.

isdir(path)

Check if a path references a directory.

Parameters:path (string) – a path in the filesystem
Return type:bool
isdirempty(path)

Check if a directory is empty (contains no files or sub-directories)

Parameters:path – a directory path
Return type:bool
isfile(path)

Check if a path references a file.

Parameters:path (string) – a path in the filesystem
Return type:bool
isvalidpath(path)

Check if a path is valid on this filesystem

Parameters:path – an fs path
listdir(path='./', wildcard=None, full=False, absolute=False, dirs_only=False, files_only=False)

Lists the the files and directories under a given path.

The directory contents are returned as a list of unicode paths.

Parameters:
  • path (string) – root of the path to list
  • wildcard (string containing a wildcard, or a callable that accepts a path and returns a boolean) – Only returns paths that match this wildcard
  • full (bool) – returns full paths (relative to the root)
  • absolute (bool) – returns absolute paths (paths beginning with /)
  • dirs_only (bool) – if True, only return directories
  • files_only (bool) – if True, only return files
Return type:

iterable of paths
Raises:
listdirinfo(path='./', wildcard=None, full=False, absolute=False, dirs_only=False, files_only=False)

Retrieves a list of paths and path info under a given path.

This method behaves like listdir() but instead of just returning the name of each item in the directory, it returns a tuple of the name and the info dict as returned by getinfo.

This method may be more efficient than calling getinfo() on each individual item returned by listdir(), particularly for network based filesystems.

Parameters:
  • path – root of the path to list
  • wildcard – filter paths that match this wildcard
  • dirs_only (bool) – only retrieve directories
  • files_only (bool) – only retrieve files
Raises:
makedir(path, recursive=False, allow_recreate=False)

Make a directory on the filesystem.

Parameters:
  • path (string) – path of directory
  • recursive (bool) – if True, any intermediate directories will also be created
  • allow_recreate – if True, re-creating a directory wont be an error
Raises:
makeopendir(path, recursive=False)

makes a directory (if it doesn’t exist) and returns an FS object for the newly created directory.

Parameters:
  • path – path to the new directory
  • recursive – if True any intermediate directories will be created
Returns:

the opened dir
Return type:

an FS object
move(src, dst, overwrite=False, chunk_size=16384)

moves a file from one location to another.

Parameters:
  • src (string) – source path
  • dst (string) – destination path
  • overwrite (bool) – When True the destination will be overwritten (if it exists), otherwise a DestinationExistsError will be thrown
  • chunk_size (integer) – Size of chunks to use when copying, if a simple copy is required
Raises:

fs.errors.DestinationExistsError – if destination exists and overwrite is False
movedir(src, dst, overwrite=False, ignore_errors=False, chunk_size=16384)

moves a directory from one location to another.

Parameters:
  • src (string) – source directory path
  • dst (string) – destination directory path
  • overwrite (bool) – if True then any existing files in the destination directory will be overwritten
  • ignore_errors (bool) – if True then this method will ignore FSError exceptions when moving files
  • chunk_size (integer) – size of chunks to use when copying, if a simple copy is required
Raises:

fs.errors.DestinationExistsError – if destination exists and overwrite is False
open(path, mode='r', buffering=-1, encoding=None, errors=None, newline=None, line_buffering=False, **kwargs)

Open a the given path as a file-like object.

Parameters:
  • path (string) – a path to file that should be opened
  • mode (string) – mode of file to open, identical to the mode string used in ‘file’ and ‘open’ builtins
  • kwargs (dict) – additional (optional) keyword parameters that may be required to open the file
Return type:

a file-like object
Raises:
opendir(path)

Opens a directory and returns a FS object representing its contents.

Parameters:path (string) – path to directory to open
Returns:the opened dir
Return type:an FS object
printtree(max_levels=5)

Prints a tree structure of the FS object to the console

Parameters:max_levels – The maximum sub-directories to display, defaults to 5. Set to None for no limit
remove(path)

Remove a file from the filesystem.

Parameters:

path (string) – Path of the resource to remove
Raises:
removedir(path, recursive=False, force=False)

Remove a directory from the filesystem

Parameters:
  • path (string) – path of the directory to remove
  • recursive (bool) – if True, empty parent directories will be removed
  • force (bool) – if True, any directory contents will be removed
Raises:
rename(src, dst)

Renames a file or directory

Parameters:
  • src (string) – path to rename
  • dst (string) – new name
Raises:
safeopen(path, mode='r', buffering=-1, encoding=None, errors=None, newline=None, line_buffering=False, **kwargs)

Like open(), but returns a NullFile if the file could not be opened.

A NullFile is a dummy file which has all the methods of a file-like object, but contains no data.

Parameters:
  • path (string) – a path to file that should be opened
  • mode (string) – mode of file to open, identical to the mode string used in ‘file’ and ‘open’ builtins
  • kwargs (dict) – additional (optional) keyword parameters that may be required to open the file
Return type:

a file-like object
setcontents(path, data='', encoding=None, errors=None, chunk_size=65536)

A convenience method to create a new file from a string or file-like object

Parameters:
  • path – a path of the file to create
  • data – a string or bytes object containing the contents for the new file
  • encoding – if data is a file open in text mode, or a text string, then use this encoding to write to the destination file
  • errors – if data is a file open in text mode or a text string, then use errors when opening the destination file
  • chunk_size – Number of bytes to read in a chunk, if the implementation has to resort to a read / copy loop
setcontents_async(path, data, encoding=None, errors=None, chunk_size=65536, progress_callback=None, finished_callback=None, error_callback=None)

Create a new file from a string or file-like object asynchronously

This method returns a threading.Event object. Call the wait method on the event object to block until all data has been written, or simply ignore it.

Parameters:
  • path – a path of the file to create
  • data – a string or a file-like object containing the contents for the new file
  • encoding – if data is a file open in text mode, or a text string, then use this encoding to write to the destination file
  • errors – if data is a file open in text mode or a text string, then use errors when opening the destination file
  • chunk_size – Number of bytes to read and write in a chunk
  • progress_callback – A function that is called periodically with the number of bytes written.
  • finished_callback – A function that is called when all data has been written
  • error_callback – A function that is called with an exception object if any error occurs during the copy process.
Returns:

An event object that is set when the copy is complete, call the wait method of this object to block until the data is written
settimes(*args, **kwds)

Set the accessed time and modified time of a file

Parameters:
  • path (string) – path to a file
  • accessed_time (datetime) – the datetime the file was accessed (defaults to current time)
  • modified_time (datetime) – the datetime the file was modified (defaults to current time)
tree(max_levels=5)

Prints a tree structure of the FS object to the console

Parameters:max_levels – The maximum sub-directories to display, defaults to 5. Set to None for no limit
validatepath(path)

Validate an fs path, throws an InvalidPathError exception if validation fails.

A path is invalid if it fails to map to a path on the underlaying filesystem. The default implementation checks for the presence of any of the characters in the meta value ‘invalid_path_chars’, but implementations may have other requirements for paths.

Parameters:path – an fs path to validatepath
Raises:fs.errors.InvalidPathError – if path does not map on to a valid path on this filesystem
walk(path='/', wildcard=None, dir_wildcard=None, search='breadth', ignore_errors=False)

Walks a directory tree and yields the root path and contents. Yields a tuple of the path of each directory and a list of its file contents.

Parameters:
  • path (string) – root path to start walking
  • wildcard (a string containing a wildcard (e.g. *.txt) or a callable that takes the file path and returns a boolean) – if given, only return files that match this wildcard
  • dir_wildcard (a string containing a wildcard (e.g. *.txt) or a callable that takes the directory name and returns a boolean) – if given, only walk directories that match the wildcard
  • search –

    a string identifying the method used to walk the directories. There are two such methods:

    • "breadth" yields paths in the top directories first
    • "depth" yields the deepest paths first
  • ignore_errors (bool) – ignore any errors reading the directory
Return type:

iterator of (current_path, paths)
walkdirs(path='/', wildcard=None, search='breadth', ignore_errors=False)

Like the ‘walk’ method but yields directories.

Parameters:
  • path (string) – root path to start walking
  • wildcard (A string containing a wildcard (e.g. *.txt) or a callable that takes the directory name and returns a boolean) – if given, only return directories that match this wildcard
  • search –

    a string identifying the method used to walk the directories. There are two such methods:

    • "breadth" yields paths in the top directories first
    • "depth" yields the deepest paths first
  • ignore_errors (bool) – ignore any errors reading the directory
Return type:

iterator of dir paths
walkfiles(path='/', wildcard=None, dir_wildcard=None, search='breadth', ignore_errors=False)

Like the ‘walk’ method, but just yields file paths.

Parameters:
  • path (string) – root path to start walking
  • wildcard (A string containing a wildcard (e.g. *.txt) or a callable that takes the file path and returns a boolean) – if given, only return files that match this wildcard
  • dir_wildcard (A string containing a wildcard (e.g. *.txt) or a callable that takes the directory name and returns a boolean) – if given, only walk directories that match the wildcard
  • search –

    a string identifying the method used to walk the directories. There are two such methods:

    • "breadth" yields paths in the top directories first
    • "depth" yields the deepest paths first
  • ignore_errors (bool) – ignore any errors reading the directory
Return type:

iterator of file paths

fs.base.SubFS

A SubFS is an FS implementation that represents a directory on another Filesystem. When you use the opendir() method it will return a SubFS instance. You should not need to instantiate a SubFS directly.

For example:

from fs.osfs import OSFS
home_fs = OSFS('foo')
bar_fs = home_fs.opendir('bar')

fs.base.NullFile

A NullFile is a file-like object with no functionality. It is used in situations where a file-like object is required but the caller doesn’t have any data to read or write.

The safeopen() method returns an NullFile instance, which can reduce error-handling code.

For example, the following code may be written to append some text to a log file:

logfile = None
try:
    logfile = myfs.open('log.txt', 'a')
    logfile.writeline('operation successful!')
finally:
    if logfile is not None:
        logfile.close()

This could be re-written using the safeopen method:

myfs.safeopen('log.txt', 'a').writeline('operation successful!')

If the file doesn’t exist then the call to writeline will be a null-operation (i.e. not do anything).