fs.base¶
This module contains the basic FS interface and a number of other essential interfaces.
fs.base.FS¶
All Filesystem objects inherit from this class.
-
class
fs.base.
FS
(thread_synchronize=True)¶ The base class for Filesystem abstraction objects. An instance of a class derived from FS is an abstraction on some kind of filesystem, such as the OS filesystem or a zip file.
The base class for Filesystem objects.
Parameters: thread_synconize – If True, a lock object will be created for the object, otherwise a dummy lock will be used. -
browse
(hide_dotfiles=False)¶ Displays the FS tree in a graphical window (requires wxPython)
Parameters: hide_dotfiles – If True, files and folders that begin with a dot will be hidden
-
cache_hint
(enabled)¶ - Recommends the use of caching. Implementations are free to use or
- ignore this value.
Parameters: enabled – If True the implementation is permitted to aggressively cache directory structure / file information. Caching such information can speed up many operations, particularly for network based filesystems. The downside of caching is that changes made to directories or files outside of this interface may not be picked up immediately.
-
cachehint
(enabled)¶ - Recommends the use of caching. Implementations are free to use or
- ignore this value.
Parameters: enabled – If True the implementation is permitted to aggressively cache directory structure / file information. Caching such information can speed up many operations, particularly for network based filesystems. The downside of caching is that changes made to directories or files outside of this interface may not be picked up immediately.
-
close
()¶ Close the filesystem. This will perform any shutdown related operations required. This method will be called automatically when the filesystem object is garbage collected, but it is good practice to call it explicitly so that any attached resourced are freed when they are no longer required.
-
copy
(src, dst, overwrite=False, chunk_size=65536)¶ Copies a file from src to dst.
Parameters: - src (string) – the source path
- dst (string) – the destination path
- overwrite (bool) – if True, then an existing file at the destination may be overwritten; If False then DestinationExistsError will be raised.
- chunk_size (bool) – size of chunks to use if a simple copy is required (defaults to 64K).
-
copydir
(src, dst, overwrite=False, ignore_errors=False, chunk_size=16384)¶ copies a directory from one location to another.
Parameters: - src (string) – source directory path
- dst (string) – destination directory path
- overwrite (bool) – if True then any existing files in the destination directory will be overwritten
- ignore_errors (bool) – if True, exceptions when copying will be ignored
- chunk_size – size of chunks to use when copying, if a simple copy is required (defaults to 16K)
-
createfile
(path, wipe=False)¶ Creates an empty file if it doesn’t exist
Parameters: - path – path to the file to create
- wipe – if True, the contents of the file will be erased
-
desc
(path)¶ Returns short descriptive text regarding a path. Intended mainly as a debugging aid.
Parameters: path – A path to describe Return type: str
-
exists
(path)¶ Check if a path references a valid resource.
Parameters: path (string) – A path in the filesystem Return type: bool
-
getcontents
(path, mode='rb', encoding=None, errors=None, newline=None)¶ Returns the contents of a file as a string.
Parameters: - path – A path of file to read
- mode – Mode to open file with (should be ‘rb’ for binary or ‘t’ for text)
- encoding – Encoding to use when reading contents in text mode
- errors – Unicode errors parameter if text mode is use
- newline – Newlines parameter for text mode decoding
Return type: str
Returns: file contents
-
getinfo
(path)¶ Returns information for a path as a dictionary. The exact content of this dictionary will vary depending on the implementation, but will likely include a few common values. The following values will be found in info dictionaries for most implementations:
- “size” - Number of bytes used to store the file or directory
- “created_time” - A datetime object containing the time the resource was created
- “accessed_time” - A datetime object containing the time the resource was last accessed
- “modified_time” - A datetime object containing the time the resource was modified
Parameters: path (string) – a path to retrieve information for
Return type: dict
Raises: - fs.errors.ParentDirectoryMissingError – if an intermediate directory is missing
- fs.errors.ResourceInvalidError – if the path is not a directory
- fs.errors.ResourceNotFoundError – if the path does not exist
-
getinfokeys
(path, *keys)¶ Get specified keys from info dict, as returned from getinfo. The returned dictionary may not contain all the keys that were asked for, if they aren’t available.
This method allows a filesystem to potentially provide a faster way of retrieving these info values if you are only interested in a subset of them.
Parameters: - path – a path to retrieve information for
- keys – the info keys you would like to retrieve
Return type: dict
-
getmeta
(meta_name, default=<class 'fs.base.NoDefaultMeta'>)¶ Retrieve a meta value associated with an FS object.
Meta values are a way for an FS implementation to report potentially useful information associated with the file system.
A meta key is a lower case string with no spaces. Meta keys may also be grouped in namespaces in a dotted notation, e.g. ‘atomic.namespaces’. FS implementations aren’t obliged to return any meta values, but the following are common:
- read_only True if the file system cannot be modified
- thread_safe True if the implementation is thread safe
- network True if the file system requires network access
- unicode_paths True if the file system supports unicode paths
- case_insensitive_paths True if the file system ignores the case of paths
- atomic.makedir True if making a directory is an atomic operation
- atomic.rename True if rename is an atomic operation, (and not implemented as a copy followed by a delete)
- atomic.setcontents True if the implementation supports setting the contents of a file as an atomic operation (without opening a file)
- free_space The free space (in bytes) available on the file system
- total_space The total space (in bytes) available on the file system
- virtual True if the filesystem defers to other filesystems
- invalid_path_chars A string containing characters that may not be used in paths
FS implementations may expose non-generic meta data through a self-named namespace. e.g.
"somefs.some_meta"
Since no meta value is guaranteed to exist, it is advisable to always supply a default value to
getmeta
.Parameters: - meta_name – The name of the meta value to retrieve
- default – An option default to return, if the meta value isn’t present
Raises: fs.errors.NoMetaError – If specified meta value is not present, and there is no default
-
getmmap
(path, read_only=False, copy=False)¶ Returns a mmap object for this path.
See http://docs.python.org/library/mmap.html for more details on the mmap module.
Parameters: - path – A path on this filesystem
- read_only – If True, the mmap may not be modified
- copy – If False then changes wont be written back to the file
Raises: fs.errors.NoMMapError – Only paths that have a syspath can be opened as a mmap
-
getpathurl
(path, allow_none=False)¶ Returns a url that corresponds to the given path, if one exists.
If the path does not have an equivalent URL form (and allow_none is False) then a
NoPathURLError
exception is thrown. Otherwise the URL will be returns as an unicode string.Parameters: - path – a path within the filesystem
- allow_none (bool) – if true, this method can return None if there is no URL form of the given path
Raises: fs.errors.NoPathURLError – If no URL form exists, and allow_none is False (the default)
Return type: unicode
-
getsize
(path)¶ Returns the size (in bytes) of a resource.
Parameters: path (string) – a path to the resource Returns: the size of the file Return type: integer
-
getsyspath
(path, allow_none=False)¶ Returns the system path (a path recognized by the OS) if one is present.
If the path does not map to a system path (and allow_none is False) then a NoSysPathError exception is thrown. Otherwise, the system path will be returned as a unicode string.
Parameters: - path – a path within the filesystem
- allow_none (bool) – if True, this method will return None when there is no system path, rather than raising NoSysPathError
Raises: fs.errors.NoSysPathError – if the path does not map on to a system path, and allow_none is set to False (default)
Return type: unicode
-
hasmeta
(meta_name)¶ Check that a meta value is supported
Parameters: meta_name – The name of a meta value to check Return type: bool
-
haspathurl
(path)¶ Check if the path has an equivalent URL form
Parameters: path – path to check Returns: True if path has a URL form Return type: bool
-
hassyspath
(path)¶ Check if the path maps to a system path (a path recognized by the OS).
Parameters: path – path to check Returns: True if path maps to a system path Return type: bool
-
ilistdir
(path='./', wildcard=None, full=False, absolute=False, dirs_only=False, files_only=False)¶ Generator yielding the files and directories under a given path.
This method behaves identically to
listdir()
but returns an generator instead of a list. Depending on the filesystem this may be more efficient than callinglistdir()
and iterating over the resulting list.
-
ilistdirinfo
(path='./', wildcard=None, full=False, absolute=False, dirs_only=False, files_only=False)¶ Generator yielding paths and path info under a given path.
This method behaves identically to
listdirinfo()
but returns an generator instead of a list. Depending on the filesystem this may be more efficient than callinglistdirinfo()
and iterating over the resulting list.
-
isdir
(path)¶ Check if a path references a directory.
Parameters: path (string) – a path in the filesystem Return type: bool
-
isdirempty
(path)¶ Check if a directory is empty (contains no files or sub-directories)
Parameters: path – a directory path Return type: bool
-
isfile
(path)¶ Check if a path references a file.
Parameters: path (string) – a path in the filesystem Return type: bool
-
isvalidpath
(path)¶ Check if a path is valid on this filesystem
Parameters: path – an fs path
-
listdir
(path='./', wildcard=None, full=False, absolute=False, dirs_only=False, files_only=False)¶ Lists the the files and directories under a given path.
The directory contents are returned as a list of unicode paths.
Parameters: - path (string) – root of the path to list
- wildcard (string containing a wildcard, or a callable that accepts a path and returns a boolean) – Only returns paths that match this wildcard
- full (bool) – returns full paths (relative to the root)
- absolute (bool) – returns absolute paths (paths beginning with /)
- dirs_only (bool) – if True, only return directories
- files_only (bool) – if True, only return files
Return type: iterable of paths
Raises: - fs.errors.ParentDirectoryMissingError – if an intermediate directory is missing
- fs.errors.ResourceInvalidError – if the path exists, but is not a directory
- fs.errors.ResourceNotFoundError – if the path is not found
-
listdirinfo
(path='./', wildcard=None, full=False, absolute=False, dirs_only=False, files_only=False)¶ Retrieves a list of paths and path info under a given path.
This method behaves like listdir() but instead of just returning the name of each item in the directory, it returns a tuple of the name and the info dict as returned by getinfo.
This method may be more efficient than calling
getinfo()
on each individual item returned bylistdir()
, particularly for network based filesystems.Parameters: - path – root of the path to list
- wildcard – filter paths that match this wildcard
- dirs_only (bool) – only retrieve directories
- files_only (bool) – only retrieve files
Raises: - fs.errors.ResourceNotFoundError – If the path is not found
- fs.errors.ResourceInvalidError – If the path exists, but is not a directory
-
makedir
(path, recursive=False, allow_recreate=False)¶ Make a directory on the filesystem.
Parameters: - path (string) – path of directory
- recursive (bool) – if True, any intermediate directories will also be created
- allow_recreate – if True, re-creating a directory wont be an error
Raises: - fs.errors.DestinationExistsError – if the path is already a directory, and allow_recreate is False
- fs.errors.ParentDirectoryMissingError – if a containing directory is missing and recursive is False
- fs.errors.ResourceInvalidError – if a path is an existing file
- fs.errors.ResourceNotFoundError – if the path is not found
-
makeopendir
(path, recursive=False)¶ makes a directory (if it doesn’t exist) and returns an FS object for the newly created directory.
Parameters: - path – path to the new directory
- recursive – if True any intermediate directories will be created
Returns: the opened dir
Return type: an FS object
-
move
(src, dst, overwrite=False, chunk_size=16384)¶ moves a file from one location to another.
Parameters: - src (string) – source path
- dst (string) – destination path
- overwrite (bool) – When True the destination will be overwritten (if it exists), otherwise a DestinationExistsError will be thrown
- chunk_size (integer) – Size of chunks to use when copying, if a simple copy is required
Raises: fs.errors.DestinationExistsError – if destination exists and overwrite is False
-
movedir
(src, dst, overwrite=False, ignore_errors=False, chunk_size=16384)¶ moves a directory from one location to another.
Parameters: - src (string) – source directory path
- dst (string) – destination directory path
- overwrite (bool) – if True then any existing files in the destination directory will be overwritten
- ignore_errors (bool) – if True then this method will ignore FSError exceptions when moving files
- chunk_size (integer) – size of chunks to use when copying, if a simple copy is required
Raises: fs.errors.DestinationExistsError – if destination exists and overwrite is False
-
open
(path, mode='r', buffering=-1, encoding=None, errors=None, newline=None, line_buffering=False, **kwargs)¶ Open a the given path as a file-like object.
Parameters: - path (string) – a path to file that should be opened
- mode (string) – mode of file to open, identical to the mode string used in ‘file’ and ‘open’ builtins
- kwargs (dict) – additional (optional) keyword parameters that may be required to open the file
Return type: a file-like object
Raises: - fs.errors.ParentDirectoryMissingError – if an intermediate directory is missing
- fs.errors.ResourceInvalidError – if an intermediate directory is an file
- fs.errors.ResourceNotFoundError – if the path is not found
-
opendir
(path)¶ Opens a directory and returns a FS object representing its contents.
Parameters: path (string) – path to directory to open Returns: the opened dir Return type: an FS object
-
printtree
(max_levels=5)¶ Prints a tree structure of the FS object to the console
Parameters: max_levels – The maximum sub-directories to display, defaults to 5. Set to None for no limit
-
remove
(path)¶ Remove a file from the filesystem.
Parameters: path (string) – Path of the resource to remove
Raises: - fs.errors.ParentDirectoryMissingError – if an intermediate directory is missing
- fs.errors.ResourceInvalidError – if the path is a directory
- fs.errors.ResourceNotFoundError – if the path does not exist
-
removedir
(path, recursive=False, force=False)¶ Remove a directory from the filesystem
Parameters: - path (string) – path of the directory to remove
- recursive (bool) – if True, empty parent directories will be removed
- force (bool) – if True, any directory contents will be removed
Raises: - fs.errors.DirectoryNotEmptyError – if the directory is not empty and force is False
- fs.errors.ParentDirectoryMissingError – if an intermediate directory is missing
- fs.errors.ResourceInvalidError – if the path is not a directory
- fs.errors.ResourceNotFoundError – if the path does not exist
-
rename
(src, dst)¶ Renames a file or directory
Parameters: - src (string) – path to rename
- dst (string) – new name
Raises: - ParentDirectoryMissingError – if a containing directory is missing
- ResourceInvalidError – if the path or a parent path is not a directory or src is a parent of dst or one of src or dst is a dir and the other don’t
- ResourceNotFoundError – if the src path does not exist
-
safeopen
(path, mode='r', buffering=-1, encoding=None, errors=None, newline=None, line_buffering=False, **kwargs)¶ Like
open()
, but returns aNullFile
if the file could not be opened.A
NullFile
is a dummy file which has all the methods of a file-like object, but contains no data.Parameters: - path (string) – a path to file that should be opened
- mode (string) – mode of file to open, identical to the mode string used in ‘file’ and ‘open’ builtins
- kwargs (dict) – additional (optional) keyword parameters that may be required to open the file
Return type: a file-like object
-
setcontents
(path, data='', encoding=None, errors=None, chunk_size=65536)¶ A convenience method to create a new file from a string or file-like object
Parameters: - path – a path of the file to create
- data – a string or bytes object containing the contents for the new file
- encoding – if data is a file open in text mode, or a text string, then use this encoding to write to the destination file
- errors – if data is a file open in text mode or a text string, then use errors when opening the destination file
- chunk_size – Number of bytes to read in a chunk, if the implementation has to resort to a read / copy loop
-
setcontents_async
(path, data, encoding=None, errors=None, chunk_size=65536, progress_callback=None, finished_callback=None, error_callback=None)¶ Create a new file from a string or file-like object asynchronously
This method returns a
threading.Event
object. Call thewait
method on the event object to block until all data has been written, or simply ignore it.Parameters: - path – a path of the file to create
- data – a string or a file-like object containing the contents for the new file
- encoding – if data is a file open in text mode, or a text string, then use this encoding to write to the destination file
- errors – if data is a file open in text mode or a text string, then use errors when opening the destination file
- chunk_size – Number of bytes to read and write in a chunk
- progress_callback – A function that is called periodically with the number of bytes written.
- finished_callback – A function that is called when all data has been written
- error_callback – A function that is called with an exception object if any error occurs during the copy process.
Returns: An event object that is set when the copy is complete, call the wait method of this object to block until the data is written
-
settimes
(*args, **kwds)¶ Set the accessed time and modified time of a file
Parameters: - path (string) – path to a file
- accessed_time (datetime) – the datetime the file was accessed (defaults to current time)
- modified_time (datetime) – the datetime the file was modified (defaults to current time)
-
tree
(max_levels=5)¶ Prints a tree structure of the FS object to the console
Parameters: max_levels – The maximum sub-directories to display, defaults to 5. Set to None for no limit
-
validatepath
(path)¶ Validate an fs path, throws an
InvalidPathError
exception if validation fails.A path is invalid if it fails to map to a path on the underlaying filesystem. The default implementation checks for the presence of any of the characters in the meta value ‘invalid_path_chars’, but implementations may have other requirements for paths.
Parameters: path – an fs path to validatepath Raises: fs.errors.InvalidPathError – if path does not map on to a valid path on this filesystem
-
walk
(path='/', wildcard=None, dir_wildcard=None, search='breadth', ignore_errors=False)¶ Walks a directory tree and yields the root path and contents. Yields a tuple of the path of each directory and a list of its file contents.
Parameters: - path (string) – root path to start walking
- wildcard (a string containing a wildcard (e.g. *.txt) or a callable that takes the file path and returns a boolean) – if given, only return files that match this wildcard
- dir_wildcard (a string containing a wildcard (e.g. *.txt) or a callable that takes the directory name and returns a boolean) – if given, only walk directories that match the wildcard
- search –
a string identifying the method used to walk the directories. There are two such methods:
"breadth"
yields paths in the top directories first"depth"
yields the deepest paths first
- ignore_errors (bool) – ignore any errors reading the directory
Return type: iterator of (current_path, paths)
-
walkdirs
(path='/', wildcard=None, search='breadth', ignore_errors=False)¶ Like the ‘walk’ method but yields directories.
Parameters: - path (string) – root path to start walking
- wildcard (A string containing a wildcard (e.g. *.txt) or a callable that takes the directory name and returns a boolean) – if given, only return directories that match this wildcard
- search –
a string identifying the method used to walk the directories. There are two such methods:
"breadth"
yields paths in the top directories first"depth"
yields the deepest paths first
- ignore_errors (bool) – ignore any errors reading the directory
Return type: iterator of dir paths
-
walkfiles
(path='/', wildcard=None, dir_wildcard=None, search='breadth', ignore_errors=False)¶ Like the ‘walk’ method, but just yields file paths.
Parameters: - path (string) – root path to start walking
- wildcard (A string containing a wildcard (e.g. *.txt) or a callable that takes the file path and returns a boolean) – if given, only return files that match this wildcard
- dir_wildcard (A string containing a wildcard (e.g. *.txt) or a callable that takes the directory name and returns a boolean) – if given, only walk directories that match the wildcard
- search –
a string identifying the method used to walk the directories. There are two such methods:
"breadth"
yields paths in the top directories first"depth"
yields the deepest paths first
- ignore_errors (bool) – ignore any errors reading the directory
Return type: iterator of file paths
-
fs.base.SubFS¶
A SubFS is an FS implementation that represents a directory on another Filesystem. When you use the opendir()
method it will return a SubFS instance. You should not need to instantiate a SubFS directly.
For example:
from fs.osfs import OSFS
home_fs = OSFS('foo')
bar_fs = home_fs.opendir('bar')
fs.base.NullFile¶
A NullFile is a file-like object with no functionality. It is used in situations where a file-like object is required but the caller doesn’t have any data to read or write.
The safeopen()
method returns an NullFile instance, which can reduce error-handling code.
For example, the following code may be written to append some text to a log file:
logfile = None
try:
logfile = myfs.open('log.txt', 'a')
logfile.writeline('operation successful!')
finally:
if logfile is not None:
logfile.close()
This could be re-written using the safeopen method:
myfs.safeopen('log.txt', 'a').writeline('operation successful!')
If the file doesn’t exist then the call to writeline will be a null-operation (i.e. not do anything).