Pre-defined types
BufReader
BufferIO.BufReader — Type
BufReader{T <: IO} <: AbstractBufReader
BufReader(io::IO, [buffer_size::Int])::BufReaderWrap an IO in a struct with a new buffer, giving it the AbstractBufReader interface.
The BufReader has an infinitely growable buffer, which may grow when fill_buffer is called.
When filling its buffer, the BufReader may move the buffered bytes to the beginning of its internal memory to make room for new bytes. This compaction is guaranteed to be amortized O(n_bytes_read), no matter how calls to fill_buffer and consume are interleaved.
This wrapper reads from io using readbytes!. If io is not EOF, and readbytes! reads zero bytes, an exception is thrown when filling the buffer.
Throw an ArgumentError if buffer_size is less than 1.
julia> rdr = BufReader(IOBuffer("Hello, world!\nabc\r\ndef"));
julia> get_buffer(rdr) |> isempty
true
julia> peek(rdr)
0x48
julia> readline(rdr)
"Hello, world!"
julia> String(readavailable(rdr))
"abc\r\ndef"Extended help
For an io::Base.IO type to function correctly when wrapped in a BufReader, it must implement:
eof(io)::Bool, which may block until at least one byte is available or EOF is known. If this returnsfalse, at least one byte must be available to read.- Either
readbytes!(io, v::MutableMemoryView{UInt8})::Integer, or elseunsafe_read(io, ::Ptr{UInt8}, ::UInt)andbytesavailable(io)::Integer. Ifreadbytes!are implemented, it should read at least one byte wheneof(io) === false.
Seeking, and filesize and position of the BufReader, additionally require io to implement seek, filesize and position.
BufWriter
BufferIO.BufWriter — Type
BufWriter{T <: IO} <: AbstractBufWriter
BufWriter(io::IO, [buffer_size::Int])::BufWriterWrap an IO in a struct with a new buffer, giving it the AbstractBufWriter interface.
The BufWriter has an infinitely growable buffer, and will only expand the buffer if grow_buffer is called on it while it does not contain any data (as shown by get_unflushed).
Throw an ArgumentError if buffer_size is < 1.
julia> io = IOBuffer(); wtr = BufWriter(io);
julia> print(wtr, "Hello!")
julia> write(wtr, [0x34, 0x12, 0x78, 0x56])
4
julia> isempty(read(io)) # wtr not flushed
true
julia> flush(wtr); seekstart(io); String(read(io))
"Hello!4\x12xV"
julia> isempty(get_unflushed(wtr))
trueExtended help
For an io::Base.IO type to function correctly when wrapped in a BufWriter, it must implement:
close(io)flush(io)unsafe_write(io, ::Ptr{UInt8}, ::UInt)
Seeking, and filesize and position of the BufWriter, additionally require io to implement seek, filesize and position, where:
- Writing overwrites existing data, and seeking does not truncate
io. filesize(io)includes any data written toiobeyondposition(io). For example,IOStreammeets this, because it writes out its own buffer when seeking.
Seeking is not supported if io is in append mode (e.g. a file opened with append = true), because then io ignores its position when writing.
IOReader
BufferIO.IOReader — Type
IOReader{T <: AbstractBufReader} <: IOWrapper type to convert an AbstractBufReader to an IO.
IOReaders implement the same part of the IO interface as AbstractBufReader, so this type is only used to satisfy type constraints.
Examples
julia> io = CursorReader("hello");
julia> f(x::IO) = String(read(x));
julia> f(io)
ERROR: MethodError: no method matching f(::CursorReader)
[...]
julia> f(IOReader(io))
"hello"IOWriter
BufferIO.IOWriter — Type
IOWriter{T <: AbstractBufWriter} <: IOWrapper type to convert an AbstractBufWriter to an IO.
IOWriters implement the same part of the IO interface as AbstractBufWriter, so this type is only used to satisfy type constraints.
Examples
julia> io = VecWriter();
julia> f(x::IO) = write(x, "hello");
julia> f(io)
ERROR: MethodError: no method matching f(::VecWriter)
[...]
julia> f(IOWriter(io))
5
julia> String(io.vec)
"hello"LineWriter
BufferIO.LineWriter — Type
LineWriter{T <: AbstractBufWriter} <: IOAn io::LineWriter wraps an inner AbstractBufWriter in an IO, and flushes when a newline is written to io.
LineWriters are useful for streams like stdout, stderr or log files, where you may want automatic flushing every time a log line has been written. Newlines are defined as 0x0a bytes, even when part of binary data (e.g. a UInt32). When a single write call has multiple newlines, LineWriter may only flush at the last newline in the write, not on every newline. LineWriter may additionally flush on non-newlines, as controlled by the inner AbstractBufWriter.
LineWriter does not take control of the inner writer. The inner writer can be obtained with the public property .inner, and may be mutated by the user without this invalidating the LineWriter.
Examples
julia> (path, io) = mktemp();
julia> writer = LineWriter(BufWriter(io));
julia> write(writer, "abc\ndef\nghi")
11
julia> read(path, String)
"abc\ndef\n"
julia> close(writer.inner) # same as `close(writer)`CursorReader
BufferIO.CursorReader — Type
CursorReader(x) <: AbstractBufReaderA seekable, stateful reader of the content of any object x which implements MemoryView(x)::MemoryView{UInt8}.
Closing it does nothing.
julia> rdr = CursorReader("some\ncontent\nhere");
julia> readline(rdr)
"some"
julia> read(rdr, String)
"content\nhere"
julia> seek(rdr, 8);
julia> read(rdr, String)
"tent\nhere"VecWriter
BufferIO.VecWriter — Type
VecWriter <: AbstractBufWriterA writer that appends to a Vector{UInt8}, which is stored in the (public) property .vec.
This type is useful as an efficient string builder through takestring!(io).
Functions flush and close do not affect the writer.
Mutating io will mutate vec and vice versa. Neither vec nor io will be invalidated by mutating the other, but doing so may affect the implicit (non-semantic) behaviour (e.g. memory reallocations or efficiency) of the other. Mutating vec invalidates buffers previously obtained from io with e.g. get_buffer.
Create with one of the following constructors:
VecWriter(): Create an empty vector.VecWriter(undef, n::Int): Create an empty vector with space for at leastnbytesVecWriter(vec::Vector{UInt8}): Append tovec, which is not copied.
A VecWriter has no notion of filesize, and cannot be seeked. Instead, resize the underlying vector io.vec.
julia> vw = VecWriter();
julia> write(vw, "Hello, world!")
13
julia> append!(vw.vec, b" More data");
julia> takestring!(vw)
"Hello, world! More data"
julia> v = collect(b"abc");
julia> write(VecWriter(v), "def");
julia> String(v)
"abcdef"ExpandReader
BufferIO.ExpandReader — Type
ExpandReader{T <: AbstractBufReader} <: AbstractBufReaderReader which wraps another reader and provides an infinitely expandable buffer. ExpandReader forwards its inner reader's buffer where possible, but if fill_buffer is called and the inner reader can't expand its buffer, ExpandReader will serve the data from its own growable buffer. This implies that fill_buffer(::ExpandReader) always returns an Int, never nothing.
ExpandReader is useful when users need to buffer a minimum amount (e.g. an entire line, or N bytes), but want to read from a generic AbstractBufReader. Generic code may use the expandable function to opt-into ExpandReader only when necessary.
ExpandReader takes control of the inner reader, and mutating the inner reader may invalidate the ExpandReader.
See also: expandable, BufferIO.is_expandable, ExpandWriter
Examples
julia> using BufferIO.Testing: BoundedReader
julia> inner = BoundedReader("Hello, world!", 4; max_fill = 4);
julia> fill_buffer(inner); fill_buffer(inner) === nothing # inner can't grow its buffer
true
julia> reader = ExpandReader(inner); # In generic code, prefer `expandable(inner)`
julia> String(get_buffer(reader)) # inner's buffer is forwarded
"Hell"
julia> while !iszero(fill_buffer(reader)) end
julia> String(get_buffer(reader)) # served from the own buffer of `reader`
"Hello, world!"
julia> BufferIO.is_expandable(typeof(reader))
trueExpandWriter
BufferIO.ExpandWriter — Type
ExpandWriter{T <: AbstractBufWriter} <: AbstractBufWriterWriter which wraps another writer and provides an infinitely expandable buffer. ExpandWriter forwards its inner writer's buffer where possible, but if grow_buffer is called and the inner writer can't expand its buffer, ExpandWriter will serve its own growable buffer. This implies that grow_buffer(::ExpandWriter) never returns zero, and get_nonempty_buffer(::ExpandWriter, ::Int) never returns nothing. If the inner writer is full, i.e. its buffer is empty and it can't make room for more data, which may happen if e.g. the underlying device cannot store more data, these functions instead throw an IOError of kind WriterFull.
ExpandWriter is useful when users need a minimum amount of contiguous buffer space (e.g. to encode a value of N bytes directly into the buffer), but want to write to a generic AbstractBufWriter. Generic code may use the expandable function to opt-into ExpandWriter only when necessary.
Data written to the own buffer of ExpandWriter is written to the inner writer when ExpandWriter is flushed, seeked or closed, or when it needs to make room in its buffer. If the inner writer becomes full before all this data could be written to it, these operations throw an IOError of kind WriterFull.
ExpandWriter takes control of the inner writer, and mutating the inner writer may invalidate the ExpandWriter.
See also: expandable, BufferIO.is_expandable, ExpandReader
Examples
julia> using BufferIO.Testing: BoundedWriter
julia> sink = UInt8[]; inner = BoundedWriter(sink, 4);
julia> get_nonempty_buffer(inner, 8) === nothing # inner can't hold 8 bytes
true
julia> writer = ExpandWriter(inner); # In generic code, prefer `expandable(inner)`
julia> buffer = get_nonempty_buffer(writer, 8); length(buffer) ≥ 8
true
julia> buffer[1:8] .= codeunits("Hello, w"); consume(writer, 8)
julia> write(writer, "orld!")
5
julia> flush(writer); String(sink)
"Hello, world!"
julia> BufferIO.is_expandable(typeof(writer))
true