fs.utils

The utils module provides a number of utility functions that don’t belong in the Filesystem interface. Generally the functions in this module work with multiple Filesystems, for instance moving and copying between non-similar Filesystems.

fs.utils.copyfile(src_fs, src_path, dst_fs, dst_path, overwrite=True, chunk_size=65536)

Copy a file from one filesystem to another. Will use system copyfile, if both files have a syspath. Otherwise file will be copied a chunk at a time.

Parameters:
  • src_fs – Source filesystem object
  • src_path – Source path
  • dst_fs – Destination filesystem object
  • dst_path – Destination path
  • chunk_size – Size of chunks to move if system copyfile is not available (default 64K)
fs.utils.movefile(src_fs, src_path, dst_fs, dst_path, overwrite=True, chunk_size=65536)

Move a file from one filesystem to another. Will use system copyfile, if both files have a syspath. Otherwise file will be copied a chunk at a time.

Parameters:
  • src_fs – Source filesystem object
  • src_path – Source path
  • dst_fs – Destination filesystem object
  • dst_path – Destination path
  • chunk_size – Size of chunks to move if system copyfile is not available (default 64K)
fs.utils.movedir(fs1, fs2, create_destination=True, ignore_errors=False, chunk_size=65536)

Moves contents of a directory from one filesystem to another.

Parameters:
  • fs1 – A tuple of (<filesystem>, <directory path>)
  • fs2 – Destination filesystem, or a tuple of (<filesystem>, <directory path>)
  • create_destination – If True, the destination will be created if it doesn’t exist
  • ignore_errors – If True, exceptions from file moves are ignored
  • chunk_size – Size of chunks to move if a simple copy is used
fs.utils.copydir(fs1, fs2, create_destination=True, ignore_errors=False, chunk_size=65536)

Copies contents of a directory from one filesystem to another.

Parameters:
  • fs1 – Source filesystem, or a tuple of (<filesystem>, <directory path>)
  • fs2 – Destination filesystem, or a tuple of (<filesystem>, <directory path>)
  • create_destination – If True, the destination will be created if it doesn’t exist
  • ignore_errors – If True, exceptions from file moves are ignored
  • chunk_size – Size of chunks to move if a simple copy is used
fs.utils.countbytes(fs)

Returns the total number of bytes contained within files in a filesystem.

Parameters:fs – A filesystem object
fs.utils.isfile(fs, path, info=None)

Check whether a path within a filesystem is a file.

If you’re able to provide the info dict for the path, this may be possible without querying the filesystem (e.g. by checking st_mode).

fs.utils.isdir(fs, path, info=None)

Check whether a path within a filesystem is a directory.

If you’re able to provide the info dict for the path, this may be possible without querying the filesystem (e.g. by checking st_mode).

fs.utils.find_duplicates(fs, compare_paths=None, quick=False, signature_chunk_size=16384, signature_size=163840)

A generator that yields the paths of duplicate files in an FS object. Files are considered identical if the contents are the same (dates or other attributes not take in to account).

Parameters:
  • fs – A filesystem object
  • compare_paths – An iterable of paths within the FS object, or all files if omitted
  • quick – If set to True, the quick method of finding duplicates will be used, which can potentially return false positives if the files have the same size and start with the same data. Do not use when deleting files!
  • signature_chunk_size – The number of bytes to read before generating a signature checksum value
  • signature_size – The total number of bytes read to generate a signature

For example, the following will list all the duplicate .jpg files in “~/Pictures”:

>>> from fs.utils import find_duplicates
>>> from fs.osfs import OSFS
>>> fs = OSFS('~/Pictures')
>>> for dups in find_duplicates(fs, fs.walkfiles('*.jpg')):
...     print list(dups)
fs.utils.print_fs(fs, path='/', max_levels=5, file_out=None, terminal_colors=None, hide_dotfiles=False, dirs_first=False, files_wildcard=None, dirs_only=False)

Prints a filesystem listing to stdout (including sub directories).

This mostly useful as a debugging aid. Be careful about printing a OSFS, or any other large filesystem. Without max_levels set, this function will traverse the entire directory tree.

For example, the following will print a tree of the files under the current working directory:

>>> from fs.osfs import *
>>> from fs.utils import *
>>> fs = OSFS('.')
>>> print_fs(fs)
Parameters:
  • fs – A filesystem object
  • path – Path of a directory to list (default “/”)
  • max_levels – Maximum levels of dirs to list (default 5), set to None for no maximum
  • file_out – File object to write output to (defaults to sys.stdout)
  • terminal_colors – If True, terminal color codes will be written, set to False for non-console output. The default (None) will select an appropriate setting for the platform.
  • hide_dotfiles – if True, files or directories beginning with ‘.’ will be removed
fs.utils.open_atomic_write(fs, path, mode='w')

Open a file for ‘atomic’ writing

This returns a context manager which ensures that a file is written in its entirety or not at all.