Pre-defined types

BufReader

BufferIO.BufReader — Type
BufReader{T <: IO} <: AbstractBufReader
BufReader(io::IO, [buffer_size::Int])::BufReader

Wrap an IO in a struct with a new buffer, giving it the AbstractBufReader interface.

The BufReader has an infinitely growable buffer, which may grow when fill_buffer is called.

When filling its buffer, the BufReader may move the buffered bytes to the beginning of its internal memory to make room for new bytes. This compaction is guaranteed to be amortized O(n_bytes_read), no matter how calls to fill_buffer and consume are interleaved.

This wrapper reads from io using readbytes!. If io is not EOF, and readbytes! reads zero bytes, an exception is thrown when filling the buffer.

Throw an ArgumentError if buffer_size is less than 1.

julia> rdr = BufReader(IOBuffer("Hello, world!\nabc\r\ndef"));

julia> get_buffer(rdr) |> isempty
true

julia> peek(rdr)
0x48

julia> readline(rdr)
"Hello, world!"

julia> String(readavailable(rdr))
"abc\r\ndef"

Extended help

For an io::Base.IO type to function correctly when wrapped in a BufReader, it must implement:

  • eof(io)::Bool, which may block until at least one byte is available or EOF is known. If this returns false, at least one byte must be available to read.
  • Either readbytes!(io, v::MutableMemoryView{UInt8})::Integer, or else unsafe_read(io, ::Ptr{UInt8}, ::UInt) and bytesavailable(io)::Integer. If readbytes! are implemented, it should read at least one byte when eof(io) === false.

Seeking, and filesize and position of the BufReader, additionally require io to implement seek, filesize and position.

BufWriter

BufferIO.BufWriter — Type
BufWriter{T <: IO} <: AbstractBufWriter
BufWriter(io::IO, [buffer_size::Int])::BufWriter

Wrap an IO in a struct with a new buffer, giving it the AbstractBufWriter interface.

The BufWriter has an infinitely growable buffer, and will only expand the buffer if grow_buffer is called on it while it does not contain any data (as shown by get_unflushed).

Throw an ArgumentError if buffer_size is < 1.

julia> io = IOBuffer(); wtr = BufWriter(io);

julia> print(wtr, "Hello!")

julia> write(wtr, [0x34, 0x12, 0x78, 0x56])
4

julia> isempty(read(io)) # wtr not flushed
true

julia> flush(wtr); seekstart(io); String(read(io))
"Hello!4\x12xV"

julia> isempty(get_unflushed(wtr))
true

Extended help

For an io::Base.IO type to function correctly when wrapped in a BufWriter, it must implement:

  • close(io)
  • flush(io)
  • unsafe_write(io, ::Ptr{UInt8}, ::UInt)

Seeking, and filesize and position of the BufWriter, additionally require io to implement seek, filesize and position, where:

  • Writing overwrites existing data, and seeking does not truncate io.
  • filesize(io) includes any data written to io beyond position(io). For example, IOStream meets this, because it writes out its own buffer when seeking.

Seeking is not supported if io is in append mode (e.g. a file opened with append = true), because then io ignores its position when writing.

IOReader

BufferIO.IOReader — Type
IOReader{T <: AbstractBufReader} <: IO

Wrapper type to convert an AbstractBufReader to an IO.

IOReaders implement the same part of the IO interface as AbstractBufReader, so this type is only used to satisfy type constraints.

Examples

julia> io = CursorReader("hello");

julia> f(x::IO) = String(read(x));

julia> f(io)
ERROR: MethodError: no method matching f(::CursorReader)
[...]

julia> f(IOReader(io))
"hello"

IOWriter

BufferIO.IOWriter — Type
IOWriter{T <: AbstractBufWriter} <: IO

Wrapper type to convert an AbstractBufWriter to an IO.

IOWriters implement the same part of the IO interface as AbstractBufWriter, so this type is only used to satisfy type constraints.

Examples

julia> io = VecWriter();

julia> f(x::IO) = write(x, "hello");

julia> f(io)
ERROR: MethodError: no method matching f(::VecWriter)
[...]

julia> f(IOWriter(io))
5

julia> String(io.vec)
"hello"

LineWriter

BufferIO.LineWriter — Type
LineWriter{T <: AbstractBufWriter} <: IO

An io::LineWriter wraps an inner AbstractBufWriter in an IO, and flushes when a newline is written to io.

LineWriters are useful for streams like stdout, stderr or log files, where you may want automatic flushing every time a log line has been written. Newlines are defined as 0x0a bytes, even when part of binary data (e.g. a UInt32). When a single write call has multiple newlines, LineWriter may only flush at the last newline in the write, not on every newline. LineWriter may additionally flush on non-newlines, as controlled by the inner AbstractBufWriter.

LineWriter does not take control of the inner writer. The inner writer can be obtained with the public property .inner, and may be mutated by the user without this invalidating the LineWriter.

Examples

julia> (path, io) = mktemp();

julia> writer = LineWriter(BufWriter(io));

julia> write(writer, "abc\ndef\nghi")
11

julia> read(path, String)
"abc\ndef\n"

julia> close(writer.inner) # same as `close(writer)`

CursorReader

BufferIO.CursorReader — Type
CursorReader(x) <: AbstractBufReader

A seekable, stateful reader of the content of any object x which implements MemoryView(x)::MemoryView{UInt8}.

Closing it does nothing.

julia> rdr = CursorReader("some\ncontent\nhere");

julia> readline(rdr)
"some"

julia> read(rdr, String)
"content\nhere"

julia> seek(rdr, 8);

julia> read(rdr, String)
"tent\nhere"

VecWriter

BufferIO.VecWriter — Type
VecWriter <: AbstractBufWriter

A writer that appends to a Vector{UInt8}, which is stored in the (public) property .vec.

This type is useful as an efficient string builder through takestring!(io).

Functions flush and close do not affect the writer.

Mutating io will mutate vec and vice versa. Neither vec nor io will be invalidated by mutating the other, but doing so may affect the implicit (non-semantic) behaviour (e.g. memory reallocations or efficiency) of the other. Mutating vec invalidates buffers previously obtained from io with e.g. get_buffer.

Create with one of the following constructors:

  • VecWriter(): Create an empty vector.
  • VecWriter(undef, n::Int): Create an empty vector with space for at least n bytes
  • VecWriter(vec::Vector{UInt8}): Append to vec, which is not copied.

A VecWriter has no notion of filesize, and cannot be seeked. Instead, resize the underlying vector io.vec.

julia> vw = VecWriter();

julia> write(vw, "Hello, world!")
13

julia> append!(vw.vec, b" More data");

julia> takestring!(vw)
"Hello, world! More data"

julia> v = collect(b"abc");

julia> write(VecWriter(v), "def");

julia> String(v)
"abcdef"

ExpandReader

BufferIO.ExpandReader — Type
ExpandReader{T <: AbstractBufReader} <: AbstractBufReader

Reader which wraps another reader and provides an infinitely expandable buffer. ExpandReader forwards its inner reader's buffer where possible, but if fill_buffer is called and the inner reader can't expand its buffer, ExpandReader will serve the data from its own growable buffer. This implies that fill_buffer(::ExpandReader) always returns an Int, never nothing.

ExpandReader is useful when users need to buffer a minimum amount (e.g. an entire line, or N bytes), but want to read from a generic AbstractBufReader. Generic code may use the expandable function to opt-into ExpandReader only when necessary.

ExpandReader takes control of the inner reader, and mutating the inner reader may invalidate the ExpandReader.

See also: expandable, BufferIO.is_expandable, ExpandWriter

Examples

julia> using BufferIO.Testing: BoundedReader

julia> inner = BoundedReader("Hello, world!", 4; max_fill = 4);

julia> fill_buffer(inner); fill_buffer(inner) === nothing # inner can't grow its buffer
true

julia> reader = ExpandReader(inner); # In generic code, prefer `expandable(inner)`

julia> String(get_buffer(reader)) # inner's buffer is forwarded
"Hell"

julia> while !iszero(fill_buffer(reader)) end

julia> String(get_buffer(reader)) # served from the own buffer of `reader`
"Hello, world!"

julia> BufferIO.is_expandable(typeof(reader))
true

ExpandWriter

BufferIO.ExpandWriter — Type
ExpandWriter{T <: AbstractBufWriter} <: AbstractBufWriter

Writer which wraps another writer and provides an infinitely expandable buffer. ExpandWriter forwards its inner writer's buffer where possible, but if grow_buffer is called and the inner writer can't expand its buffer, ExpandWriter will serve its own growable buffer. This implies that grow_buffer(::ExpandWriter) never returns zero, and get_nonempty_buffer(::ExpandWriter, ::Int) never returns nothing. If the inner writer is full, i.e. its buffer is empty and it can't make room for more data, which may happen if e.g. the underlying device cannot store more data, these functions instead throw an IOError of kind WriterFull.

ExpandWriter is useful when users need a minimum amount of contiguous buffer space (e.g. to encode a value of N bytes directly into the buffer), but want to write to a generic AbstractBufWriter. Generic code may use the expandable function to opt-into ExpandWriter only when necessary.

Data written to the own buffer of ExpandWriter is written to the inner writer when ExpandWriter is flushed, seeked or closed, or when it needs to make room in its buffer. If the inner writer becomes full before all this data could be written to it, these operations throw an IOError of kind WriterFull.

ExpandWriter takes control of the inner writer, and mutating the inner writer may invalidate the ExpandWriter.

See also: expandable, BufferIO.is_expandable, ExpandReader

Examples

julia> using BufferIO.Testing: BoundedWriter

julia> sink = UInt8[]; inner = BoundedWriter(sink, 4);

julia> get_nonempty_buffer(inner, 8) === nothing # inner can't hold 8 bytes
true

julia> writer = ExpandWriter(inner); # In generic code, prefer `expandable(inner)`

julia> buffer = get_nonempty_buffer(writer, 8); length(buffer) ≥ 8
true

julia> buffer[1:8] .= codeunits("Hello, w"); consume(writer, 8)

julia> write(writer, "orld!")
5

julia> flush(writer); String(sink)
"Hello, world!"

julia> BufferIO.is_expandable(typeof(writer))
true