fs.filelike

This module takes care of the groundwork for implementing and manipulating objects that provide a rich file-like interface, including reading, writing, seeking and iteration.

The main class is FileLikeBase, which implements the entire file-like interface on top of primitive _read(), _write(), _seek(), _tell() and _truncate() methods. Subclasses may implement any or all of these methods to obtain the related higher-level file behaviors.

Other useful classes include:

  • StringIO: a version of the builtin StringIO class, patched to more
    closely preserve the semantics of a standard file.
  • FileWrapper: a generic base class for wrappers around a filelike object
    (think e.g. compression or decryption).
  • SpooledTemporaryFile: a version of the builtin SpooledTemporaryFile
    class, patched to more closely preserve the semantics of a standard file.
  • LimitBytesFile: a filelike wrapper that limits the total bytes read
    from a file; useful for turning a socket into a file without reading past end-of-data.
class fs.filelike.FileLikeBase(bufsize=65536)

Base class for implementing file-like objects.

This class takes a lot of the legwork out of writing file-like objects with a rich interface. It implements the higher-level file-like methods on top of five primitive methods: _read, _write, _seek, _tell and _truncate. See their docstrings for precise details on how these methods behave.

Subclasses then need only implement some subset of these methods for rich file-like interface compatibility. They may of course override other methods as desired.

The class is missing the following attributes and methods, which don’t really make sense for anything but real files:

  • fileno()
  • isatty()
  • encoding
  • mode
  • name
  • newlines

Unlike standard file objects, all read methods share the same buffer and so can be freely mixed (e.g. read(), readline(), next(), ...).

This class understands and will accept the following mode strings, with any additional characters being ignored:

  • r - open the file for reading only.
  • r+ - open the file for reading and writing.
  • r- - open the file for streamed reading; do not allow seek/tell.
  • w - open the file for writing only; create the file if
    it doesn’t exist; truncate it to zero length.
  • w+ - open the file for reading and writing; create the file
    if it doesn’t exist; truncate it to zero length.
  • w- - open the file for streamed writing; do not allow seek/tell.
  • a - open the file for writing only; create the file if it
    doesn’t exist; place pointer at end of file.
  • a+ - open the file for reading and writing; create the file
    if it doesn’t exist; place pointer at end of file.

These are mostly standard except for the “-” indicator, which has been added for efficiency purposes in cases where seeking can be expensive to simulate (e.g. compressed files). Note that any file opened for both reading and writing must also support seeking.

FileLikeBase Constructor.

The optional argument ‘bufsize’ specifies the number of bytes to read at a time when looking for a newline character. Setting this to a larger number when lines are long should improve efficiency.

close()

Flush write buffers and close the file.

The file may not be accessed further once it is closed.

flush()

Flush internal write buffer, if necessary.

next()

next() method complying with the iterator protocol.

File-like objects are their own iterators, with each call to next() returning subsequent lines from the file.

read(size=-1)

Read at most ‘size’ bytes from the file.

Bytes are returned as a string. If ‘size’ is negative, zero or missing, the remainder of the file is read. If EOF is encountered immediately, the empty string is returned.

readline(size=-1)

Read a line from the file, or at most <size> bytes.

readlines(sizehint=-1)

Return a list of all lines in the file.

seek(offset, whence=0)

Move the internal file pointer to the given location.

tell()

Determine current position of internal file pointer.

truncate(size=None)

Truncate the file to the given size.

If <size> is not specified or is None, the current file position is used. Note that this method may fail at runtime if the underlying filelike object is not truncatable.

write(string)

Write the given string to the file.

writelines(seq)

Write a sequence of lines to the file.

xreadlines()

Iterator over lines in the file - equivalent to iter(self).

class fs.filelike.FileWrapper(wrapped_file, mode=None)

Base class for objects that wrap a file-like object.

This class provides basic functionality for implementing file-like objects that wrap another file-like object to alter its functionality in some way. It takes care of house-keeping duties such as flushing and closing the wrapped file.

Access to the wrapped file is given by the attribute wrapped_file. By convention, the subclass’s constructor should accept this as its first argument and pass it to its superclass’s constructor in the same position.

This class provides a basic implementation of _read() and _write() which just calls read() and write() on the wrapped object. Subclasses will probably want to override these.

FileWrapper constructor.

‘wrapped_file’ must be a file-like object, which is to be wrapped in another file-like object to provide additional functionality.

If given, ‘mode’ must be the access mode string under which the wrapped file is to be accessed. If not given or None, it is looked up on the wrapped file if possible. Otherwise, it is not set on the object.

close()

Close the object for reading/writing.

flush()

Flush the write buffers of the file.

class fs.filelike.LimitBytesFile(size, fileobj, *args, **kwds)

Filelike wrapper to limit bytes read from a stream.

class fs.filelike.SpooledTemporaryFile(max_size=0, mode='w+b', bufsize=-1, *args, **kwds)

SpooledTemporaryFile wrapper with some compatibility fixes.

This is a simple compatibility wrapper around the native class of the same name, fixing some corner-cases of its behavior. Specifically:

  • have truncate() accept a size argument
  • roll to disk is seeking past the max in-memory size
  • use improved StringIO class from this module
class fs.filelike.StringIO(data=None, mode=None)

StringIO wrapper that more closely matches standard file behavior.

This is a simple compatibility wrapper around the native StringIO class which fixes some corner-cases of its behavior. Specifically:

  • adding __enter__ and __exit__ methods
  • having truncate(size) zero-fill when growing the file