filesystem

Filesystem operations on a linked LucidLink filespace.

Provides file, directory, metadata, and locking operations. Accessed via filespace.fs after linking to a filespace.

class lucidlink.filesystem.FileHandle(filesystem: Filesystem, path: str, mode: str)[source]

Bases: object

Context manager for file handles (legacy API).

Provides Pythonic file operations with automatic handle cleanup. Use with ‘with’ statement to ensure handles are properly closed.

Example

with fs.open_legacy("/path/to/file.txt", "r") as fh:
    data = fh.read()
# Handle automatically closed
close() → None[source]

Close the file handle. Safe to call multiple times.

property closed: bool

Check if the file handle is closed.

property mode: str

Get the file open mode.

property path: str

Get the file path.

read(size: int = -1, offset: int = 0) → bytes[source]

Read data from the file.

Parameters:

  • size – Number of bytes to read (-1 = read entire file)

  • offset – Byte offset to start reading from

Returns:

Bytes read from the file

write(data: bytes, offset: int = 0) → None[source]

Write data to the file.

Parameters:

  • data – Bytes to write to the file

  • offset – Byte offset to start writing at

class lucidlink.filesystem.Filesystem[source]

Bases: object

Filesystem operations on a linked LucidLink filespace.

Provides file, directory, metadata, and locking operations. Obtained via the filespace.fs property after linking.

Example

filespace = workspace.link_filespace(name="production-data")
entries = filespace.fs.read_dir("/")
filespace.fs.create_dir("/new-folder")
with filespace.fs.open("/file.txt", "wb") as f:
    f.write(b"data")
create(path: str) → int[source]

Create a new file and return a handle ID.

Use open() with mode=”w” for most cases. This is a low-level API.

Parameters:

path – File path to create

Returns:

File handle ID (use with read/write/close methods)

Raises:

  • FileExistsError – If file already exists

  • PermissionError – If no create permission

create_dir(path: str) → None[source]

Create a directory.

Creates parent directories if they don’t exist (like mkdir -p).

Parameters:

path – Directory path to create

Raises:

  • FileExistsError – If directory already exists

  • PermissionError – If no create permission

delete(path: str) → None[source]

Delete a file.

Parameters:

path – File path to delete

Raises:

  • FileNotFoundError – If file doesn’t exist

  • IsADirectoryError – If path is a directory

  • PermissionError – If no delete permission

delete_dir(path: str, recursive: bool = False) → None[source]

Delete a directory.

Parameters:

  • path – Directory path to delete

  • recursive – If True, delete non-empty directories (default: False)

Raises:

  • FileNotFoundError – If directory doesn’t exist

  • NotADirectoryError – If path is not a directory

  • OSError – If directory is not empty and recursive=False

  • PermissionError – If no delete permission

dir_exists(path: str) → bool[source]

Check if a directory exists.

Parameters:

path – Directory path to check

Returns:

True if directory exists, False otherwise

file_exists(path: str) → bool[source]

Check if a file exists.

Parameters:

path – File path to check

Returns:

True if file exists, False otherwise

get_entry(path: str) → DirEntry[source]

Get metadata for a file or directory.

Parameters:

path – Path to get metadata for

Returns:

DirEntry with entry information

Raises:

FileNotFoundError – If path doesn’t exist

get_size() → FilespaceSize[source]

Get filespace size information.

Returns:

FilespaceSize with entries, data, storage, and external file info

Raises:

RuntimeError – If operation fails

get_statistics() → FilespaceStatistics[source]

Get filespace statistics.

Returns:

FilespaceStatistics with file/directory counts and size info

Raises:

RuntimeError – If operation fails

list_dir(path: str) → List[str][source]

List directory contents, returning just filenames (convenience method).

Parameters:

path – Directory path to list

Returns:

List of filenames (not full paths)

Raises:

  • NotADirectoryError – If path is not a directory

  • FileNotFoundError – If directory doesn’t exist

lock_byte_range(handle_id: int, offset: int, length: int, lock_type: str = 'exclusive', blocking: bool = True) → bool[source]

Lock a byte range of an open file.

Acquires a lock on the specified byte range. This provides cross-daemon file locking that is coordinated through the LucidHub, ensuring proper mutual exclusion across all clients accessing the filespace.

Parameters:

  • handle_id – File handle ID from a low-level open operation

  • offset – Start offset of the byte range to lock (0 for whole file)

  • length – Length of the byte range to lock (use file size for whole file)

  • lock_type – Lock type (default: “exclusive”): - “exclusive”: Exclusive lock (no other locks allowed) - “shared” or “read”: Shared read lock (multiple readers allowed) - “write”: Protected write lock

  • blocking – If True (default), wait until lock is available. If False, return immediately if lock unavailable.

Returns:

True if lock was acquired, False if non-blocking and lock unavailable

Raises:

  • RuntimeError – If handle is invalid or lock operation fails

  • ValueError – If lock_type is invalid

Example

handle_id = filespace.fs._native.open("/data.db", "r+b")
try:
    if filespace.fs.lock_byte_range(handle_id, 0, 1, "exclusive"):
        # Perform exclusive operations
        pass
finally:
    filespace.fs.unlock_byte_range(handle_id, 0, 1)
    filespace.fs._native.close(handle_id)
move(src: str, dst: str) → None[source]

Move/rename a file or directory.

Parameters:

  • src – Source path

  • dst – Destination path

Raises:

  • FileNotFoundError – If source doesn’t exist

  • FileExistsError – If destination already exists

  • PermissionError – If no move permission

open(path: str, mode: str = 'rb', buffering: int = -1, encoding: str | None = None, errors: str | None = None, newline: str | None = None, lock_type: str = '') → LucidFileStream | BufferedReader | BufferedWriter | TextIOWrapper[source]

Open a file with streaming support.

Returns an io.RawIOBase-compatible stream that works with standard Python libraries and third-party packages (Pandas, LangChain, PyTorch, etc.).

Parameters:

  • path – File path to open

  • mode – Open mode (default: ‘rb’): - ‘r’: Read text (default encoding: utf-8) - ‘w’: Write text (create/truncate, default encoding: utf-8) - ‘a’: Append text (default encoding: utf-8) - ‘r+’: Read/write text (default encoding: utf-8) - ‘w+’: Write/read text (create/truncate, default encoding: utf-8) - ‘rb’: Read binary - ‘wb’: Write binary (create/truncate) - ‘ab’: Append binary - ‘r+b’: Read/write binary - ‘w+b’: Write/read binary (create/truncate) - ‘a+b’: Append/read binary - ‘rt’: Read text (explicit, same as ‘r’) - ‘wt’: Write text (explicit, same as ‘w’)

  • buffering – Buffer size: - -1 (default): Use system default (8192 bytes) - 0: Unbuffered (binary modes only) - 1: Line buffered (text mode only) - >1: Buffer size in bytes

  • encoding – Text encoding (e.g., ‘utf-8’, ‘latin-1’) Required for text modes (‘r’, ‘w’ without ‘b’)

  • errors – Error handling (‘strict’, ‘ignore’, ‘replace’)

  • newline – Newline handling (None, ‘’, ‘n’, ‘r’, ‘rn’)

  • lock_type – Lock type - “” (no lock), “shared” (read), “exclusive” (write). Lock is held for lifetime of file handle and released on close.

Returns:

File stream object supporting read, write, seek, tell operations. Compatible with context managers (with statement).

Example

# Binary mode
with filespace.fs.open("/file.dat", "rb") as f:
    data = f.read()

# Text mode
with filespace.fs.open("/file.txt", "rt", encoding="utf-8") as f:
    for line in f:
        print(line.strip())

# With Pandas
with filespace.fs.open("/data.csv", "rb") as f:
    df = pd.read_csv(f)

# Writing
with filespace.fs.open("/output.txt", "wb") as f:
    f.write(b"Hello, LucidLink!")

# With exclusive locking (SQLite-style)
with filespace.fs.open("/db.sqlite", "r+b", lock_type="exclusive") as f:
    data = f.read()
Raises:

  • FileNotFoundError – If file doesn’t exist (read mode)

  • PermissionError – If no access permission

  • ValueError – If mode is invalid

open_legacy(path: str, mode: str = 'r') → FileHandle[source]

Open a file using legacy FileHandle API (deprecated).

This method is preserved for backward compatibility. New code should use open() instead, which returns a more capable io.RawIOBase stream.

Parameters:

  • path – File path to open

  • mode – Open mode (‘r’, ‘w’, ‘a’)

Returns:

FileHandle context manager

Raises:

  • FileNotFoundError – If file doesn’t exist (read mode)

  • PermissionError – If no access permission

read_dir(path: str) → List[DirEntry][source]

List directory contents.

Parameters:

path – Directory path to list

Returns:

List of DirEntry objects with file entry information.

Raises:

  • NotADirectoryError – If path is not a directory

  • FileNotFoundError – If directory doesn’t exist

  • PermissionError – If no read permission

read_file(path: str) → bytes[source]

Read entire file contents (convenience method).

Parameters:

path – File path to read

Returns:

File contents as bytes

Raises:

  • FileNotFoundError – If file doesn’t exist

  • PermissionError – If no read permission

truncate(path: str, size: int) → None[source]

Truncate or extend file to specified size.

Parameters:

  • path – File path to truncate

  • size – New file size in bytes

Raises:

  • FileNotFoundError – If file doesn’t exist

  • PermissionError – If no write permission

  • RuntimeError – If truncation fails

unlock_all_byte_ranges(handle_id: int) → None[source]

Unlock all byte ranges on an open file.

Releases all previously acquired locks on the file.

Parameters:

handle_id – File handle ID from a low-level open operation

Raises:

RuntimeError – If handle is invalid or unlock fails

unlock_byte_range(handle_id: int, offset: int, length: int) → None[source]

Unlock a byte range of an open file.

Releases a previously acquired lock on the specified byte range.

Parameters:

  • handle_id – File handle ID from a low-level open operation

  • offset – Start offset of the byte range to unlock

  • length – Length of the byte range to unlock

Raises:

RuntimeError – If handle is invalid or unlock fails

write_file(path: str, data: bytes) → None[source]

Write data to file, creating it if needed (convenience method).

Parameters:

  • path – File path to write

  • data – Bytes to write

Raises:

  • PermissionError – If no permission to write file

  • TypeError – If data is not bytes