fastavro.write

writer(fo, schema, records, codec='null', sync_interval=16000, metadata=None, validator=None, sync_marker=None)

Write records to fo (stream) according to schema

Parameters:
  • fo (file-like) – Output stream
  • schema (dict) – Writer schema
  • records (iterable) – Records to write. This is commonly a list of the dictionary representation of the records, but it can be any iterable
  • codec (string, optional) – Compression codec, can be ‘null’, ‘deflate’ or ‘snappy’ (if installed)
  • sync_interval (int, optional) – Size of sync interval
  • metadata (dict, optional) – Header metadata
  • validator (None, True or a function) – Validator function. If None (the default) - no validation. If True then then fastavro.validation.validate will be used. If it’s a function, it should have the same signature as fastavro.writer.validate and raise an exeption on error.
  • sync_marker (bytes, optional) – A byte string used as the avro sync marker. If not provided, a random byte string will be used.

Example:

from fastavro import writer, parse_schema

schema = {
    'doc': 'A weather reading.',
    'name': 'Weather',
    'namespace': 'test',
    'type': 'record',
    'fields': [
        {'name': 'station', 'type': 'string'},
        {'name': 'time', 'type': 'long'},
        {'name': 'temp', 'type': 'int'},
    ],
}
parsed_schema = parse_schema(schema)

records = [
    {u'station': u'011990-99999', u'temp': 0, u'time': 1433269388},
    {u'station': u'011990-99999', u'temp': 22, u'time': 1433270389},
    {u'station': u'011990-99999', u'temp': -11, u'time': 1433273379},
    {u'station': u'012650-99999', u'temp': 111, u'time': 1433275478},
]

with open('weather.avro', 'wb') as out:
    writer(out, parsed_schema, records)

Given an existing avro file, it’s possible to append to it by re-opening the file in a+b mode. If the file is only opened in ab mode, we aren’t able to read some of the existing header information and an error will be raised. For example:

# Write initial records
with open('weather.avro', 'wb') as out:
    writer(out, parsed_schema, records)

# Write some more records
with open('weather.avro', 'a+b') as out:
    writer(out, parsed_schema, more_records)
schemaless_writer(fo, schema, record)

Write a single record without the schema or header information

Parameters:
  • fo (file-like) – Output file
  • schema (dict) – Schema
  • record (dict) – Record to write

Example:

parsed_schema = fastavro.parse_schema(schema)
with open('file.avro', 'rb') as fp:
    fastavro.schemaless_writer(fp, parsed_schema, record)

Note: The schemaless_writer can only write a single record.