stream

Streaming file interface for LucidLink files.

Provides io.RawIOBase-compatible file streams for seamless integration with Python’s standard library and third-party libraries (Pandas, LangChain, PyTorch, etc.).

class lucidlink.stream.LucidFileStream(fs_wrapper: Any, path: str, mode: str = 'rb', lock_type: str = '')[source]

Bases: RawIOBase

Streaming file access for LucidLink files.

Implements Python’s io.RawIOBase interface for efficient, streaming access to files in LucidLink filespaces. Compatible with standard Python I/O operations and third-party libraries that expect file-like objects.

This class provides: - Standard file operations: read, write, seek, tell, truncate - Context manager support (with statement) - Iterator support (for line in file) - Optional file locking (on-open locking) - Compatibility with: Pandas, LangChain, PyTorch, Hugging Face, etc.

Parameters:

fs_wrapper – PythonFileSystemWrapper instance (from lucidlink_native)
path – File path within the filespace
mode – File open mode (‘r’, ‘rb’, ‘w’, ‘wb’, ‘a’, ‘ab’, ‘r+’, ‘r+b’)
lock_type – Lock type - “” (no lock), “shared” (read), “exclusive” (write)

Example

with filespace.fs.open("/data/file.txt", "rb") as f:
    data = f.read()

with filespace.fs.open("/data/output.csv", "wb") as f:
    f.write(b"column1,column2\n")
    f.write(b"value1,value2\n")

# With exclusive locking for SQLite-style access
with filespace.fs.open("/data/db.sqlite", "r+b", lock_type="exclusive") as f:
    data = f.read()

close() → None[source]: Close the stream and release resources.

property closed: bool

Return whether the stream is closed.

Returns:: True if stream is closed

fileno() → int[source]

Return underlying file descriptor.

Raises:: io.UnsupportedOperation – LucidLink files don’t have OS file descriptors

isatty() → bool[source]

Return whether this is an interactive stream.

Returns:: False (LucidLink files are never TTY devices)

property mode: str

Return the file mode.

Returns:: Mode string used to open the file

property name: str

Return the file path.

Returns:: Path to file within filespace

read(size: int = -1) → bytes[source]

Read up to size bytes from the stream.

Parameters:

size – Number of bytes to read, or -1 to read entire file

Returns:

Bytes read from file

Raises:

ValueError – If stream is closed
io.UnsupportedOperation – If stream not opened for reading

readable() → bool[source]

Return whether the stream supports reading.

Returns:: True if stream was opened for reading

readinto(b: bytearray) → int | None[source]

Read bytes into a pre-allocated buffer.

Parameters:: b – Buffer to read into
Returns:: Number of bytes read, or None if EOF

seek(offset: int, whence: int = 0) → int[source]

Change stream position.

Parameters:

offset – Offset in bytes
whence – Reference point (0=start, 1=current, 2=end)

Returns:

New absolute position

seekable() → bool[source]

Return whether the stream supports random access.

Returns:: True (LucidLink files always support seeking)

tell() → int[source]

Return current stream position.

Returns:: Current position in bytes

truncate(size: int | None = None) → int[source]

Resize the stream to the given size.

Parameters:

size – New size in bytes, or current position if None

Returns:

New size

Raises:

ValueError – If stream is closed
io.UnsupportedOperation – If stream not opened for writing

writable() → bool[source]

Return whether the stream supports writing.

Returns:: True if stream was opened for writing

write(b: bytes) → int[source]

Write bytes to the stream.

Parameters:

b – Bytes to write

Returns:

Number of bytes written

Raises:

ValueError – If stream is closed
io.UnsupportedOperation – If stream not opened for writing

lucidlink.stream.open_buffered(fs_wrapper: Any, path: str, mode: str = 'rb', buffer_size: int = 131072, lock_type: str = '') → BufferedIOBase[source]

Open a file with buffering for improved performance.

Parameters:

fs_wrapper – PythonFileSystemWrapper from lucidlink_native module
path – Path to file within filespace
mode – File open mode (default: ‘rb’)
buffer_size – Buffer size in bytes (default: 8192)
lock_type – Lock type - “” (no lock), “shared” (read), “exclusive” (write)

Returns:

BufferedReader, BufferedWriter, or BufferedRandom wrapping LucidFileStream

Example

with open_buffered(fs, "/data/large.csv", "rb") as f:
    for line in f:
        process(line)

lucidlink.stream.open_text(fs_wrapper: Any, path: str, mode: str = 'r', encoding: str = 'utf-8', errors: str = 'strict', newline: str | None = None, lock_type: str = '') → TextIOWrapper[source]

Open a file in text mode.

Parameters:

fs_wrapper – PythonFileSystemWrapper from lucidlink_native module
path – Path to file within filespace
mode – File open mode (default: ‘r’)
encoding – Text encoding (default: ‘utf-8’)
errors – Error handling strategy (default: ‘strict’)
newline – Newline handling (None, ‘’, ‘n’, ‘r’, ‘rn’)
lock_type – Lock type - “” (no lock), “shared” (read), “exclusive” (write)

Returns:

TextIOWrapper for text-mode access

Example

with open_text(fs, "/data/report.txt", "r") as f:
    for line in f:
        print(line.strip())