AbstractBufWriter

Core, low-level interface

Similar to AbstractBufReader, the core interface of AbstractBufWriter consists of three functions:

  • get_buffer(io) returns a mutable view into the part of the buffer which is yet unused. Data is written to io by copying to the first bytes of the buffer, then calling consume.
  • grow_buffer(io) request to expand the buffer returned by future calls to get_buffer. This may happen by flushing data in the buffer or by reallocating a larger buffer
  • consume(io, n::Int) signals that the first n bytes in the buffer are written to io, and will therefore not be returned from future calls to get_buffer.

Example: Writing v::Vector{UInt8} to an AbstractBufWriter

The method write(::AbstractBufWriter, v::Vector{UInt8}) is already implemented, but it's illustrative to see how this be implemented in terms of the core primitives above.

First, let's define it in terms of ImmutableMemoryView, and then forward the Vector method to the memory one.

using MemoryViews

my_write(io::AbstractBufWriter, v::Vector{UInt8}) = my_write(io, ImmutableMemoryView(v))

function my_write(io::AbstractBufWriter, mem::ImmutableMemoryView{UInt8})::Int
    n_bytes = length(mem)
    while !isempty(mem)
        # Get mutable buffer with uninitialized data to write to
        buffer = get_buffer(io)::MutableMemoryView{UInt8}
        if isempty(buffer)
            # grow_buffer cannot return `nothing`, unlike for readers, but the writer
            # may still be unable to add more bytes (in which case grow_buffer returns
            # zero). A real implementation would use a better error
            iszero(grow_buffer(io)) && error("Could not flush")
            buffer = get_buffer(io)::MutableMemoryView{UInt8}
        end
        mn = min(length(mem), length(buffer))
        @assert !iszero(mn)
        (to_write, mem) = @inbounds split_at(mem, mn + 1)
        @inbounds copyto!(@inbounds(buffer[1:mn]), to_write)
        # Mark the first `mn` bytes of the buffer as being committed, thereby
        # actually writing it to `io`
        @inbounds consume(io, mn)
    end
    return n_bytes
end
BufIO.get_buffer — Function
get_buffer(io::AbstractBufReader)::ImmutableMemoryView{UInt8}

Get the available bytes of io.

Calling this function when the buffer is empty should do actual system I/O, and in particular should not attempt to fill the buffer. To fill the buffer, call fill_buffer.

get_buffer(io::AbstractBufWriter)::MutableMemoryView{UInt8}

Get the available mutable buffer of io that can be written to.

Calling this function should do actual system I/O, and in particular should not attempt to flush data from the buffer. To increase the size of the buffer, call grow_buffer.

BufIO.grow_buffer — Function
grow_buffer(io::AbstractBufWriter)::Int

Increase the amount of bytes in the writeable buffer of io if possible, returning the number of bytes added. After calling grow_buffer and getting n, the buffer obtained by get_buffer should have n more bytes.

  • If there is data in the buffer of io, flush it to the underlying io if possible.
  • Else, if io's buffer can be expanded, do so.
  • Else, return zero
Note

Idiomatically, users should not call grow_buffer when the buffer is not empty, because doing so forces growing the buffer instead of letting io choose an optimal buffer size. Calling grow_buffer with a nonempty buffer is only appropriate if, for algorithmic reasons you need io buffer to be able to hold some minimum amount of data before flushing.

BufIO.consume — Function
consume(io::Union{AbstractBufReader, AbstractBufWriter}, n::Int)::Nothing

Remove the first n bytes of the buffer of io. Consumed bytes will not be returned by future calls to get_buffer.

If n is negative, or larger than the current buffer size, throw an IOError with ConsumeBufferError kind. This check is a boundscheck and may be elided with @inbounds.

Notable AbstractWriter functions

BufIO.get_unflushed — Function
get_unflushed(io::AbstractBufWriter)::MutableMemoryView{UInt8}

Return a view into the buffered data already written to io and consumed, but not yet flushed to its underlying IO.

Bytes not appearing in the buffer may not be completely flushed if there are more layers of buffering in the IO wrapped by io. However, any bytes already consumed and not returned in get_unflushed should not be buffered in io.

Mutating the returned buffer is allowed, and should not cause io to malfunction. When io flushes, the updated values in the buffer will be flushed.

This function has no default implementation and methods are optionally added to subtypes of AbstractBufWriter that can fullfil the above restrictions.

BufIO.get_nonempty_buffer — Method
get_nonempty_buffer(
    io::AbstractBufWriter, min_size::Int
)::Union{Nothing, MutableMemoryView{UInt8}}

Get a nonempty buffer of at least size min_size.

This function need not be implemented for subtypes of AbstractBufWriter that do not flush their writes to an underlying IO.

Warning

Use of this functions may cause excessive buffering without flushing, which is less efficient than calling the one-argument method in a loop. Authors should avoid implementing this method for types capable of flushing.