.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/17_compression_framework.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_17_compression_framework.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_17_compression_framework.py:


Compression Framework
=====================

This example shows the intended lifecycle:

* write a model with the fast/default compression path
* inspect the persisted compression metadata
* promote the same model in place with ``compress()``

.. GENERATED FROM PYTHON SOURCE LINES 11-22

.. code-block:: Python


    from pathlib import Path
    import json
    import tempfile

    import pyarrow.parquet as pq

    from parq_blockmodel import ParquetBlockModel

    # sphinx_gallery_thumbnail_path = "../docs/_static/branding/parq-blockmodel-gallery-thumbnail.svg"








.. GENERATED FROM PYTHON SOURCE LINES 23-24

Create a small demo block model.

.. GENERATED FROM PYTHON SOURCE LINES 24-31

.. code-block:: Python


    temp_dir = Path(tempfile.gettempdir()) / "compression_framework_example"
    temp_dir.mkdir(parents=True, exist_ok=True)

    pbm = ParquetBlockModel.create_demo_block_model(temp_dir / "compression_demo.parquet")
    pbm





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    ParquetBlockModel(name=compression_demo, path=/tmp/compression_framework_example/compression_demo.pbm)



.. GENERATED FROM PYTHON SOURCE LINES 32-33

Inspect the active-write compression metadata.

.. GENERATED FROM PYTHON SOURCE LINES 33-37

.. code-block:: Python


    before = pq.read_metadata(pbm.blockmodel_path).metadata or {}
    json.loads(before[b"parq-blockmodel"].decode("utf-8"))





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    {'geometry': {'schema_version': '1.1', 'corner': [0.0, 0.0, 0.0], 'block_size': [1.0, 1.0, 1.0], 'shape': [3, 3, 3], 'origin': [0.0, 0.0, 0.0], 'axis_u': [1.0, 0.0, 0.0], 'axis_v': [0.0, 1.0, 0.0], 'axis_w': [0.0, 0.0, 1.0], 'srs': None, 'world_id_encoding': {'enabled': True, 'column': 'world_id', 'frame': 'world_xyz', 'axis_order': ['x', 'y', 'z'], 'encoding': {'type': 'morton_zorder', 'version': '1.0', 'bits_per_axis': 24, 'axis_bits': {'x': 24, 'y': 24, 'z': 16}, 'signed': False}, 'quantization': {'scale': 10.0, 'rounding': 'nearest', 'precision_decimals': 1}, 'offset': {'x': 0.0, 'y': 0.0, 'z': 0.0, 'units': 'world'}, 'range_after_offset': {'x_min': 0.0, 'x_max': 1677721.5, 'y_min': 0.0, 'y_max': 1677721.5, 'z_min': 0.0, 'z_max': 6553.5}, 'overflow_policy': 'error', 'null_policy': 'no_nulls'}}, 'compression': {'mode': 'active', 'default': {'codec': 'snappy', 'level': None}, 'columns': {}}}



.. GENERATED FROM PYTHON SOURCE LINES 38-39

Rewrite the model in place using archive compression.

.. GENERATED FROM PYTHON SOURCE LINES 39-42

.. code-block:: Python


    pbm.compress(level=7)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    ParquetBlockModel(name=compression_demo, path=/tmp/compression_framework_example/compression_demo.pbm)



.. GENERATED FROM PYTHON SOURCE LINES 43-44

Check the archived policy after the rewrite.

.. GENERATED FROM PYTHON SOURCE LINES 44-47

.. code-block:: Python


    after = pq.read_metadata(pbm.blockmodel_path).metadata or {}
    json.loads(after[b"parq-blockmodel"].decode("utf-8"))




.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    {'geometry': {'schema_version': '1.1', 'corner': [0.0, 0.0, 0.0], 'block_size': [1.0, 1.0, 1.0], 'shape': [3, 3, 3], 'origin': [0.0, 0.0, 0.0], 'axis_u': [1.0, 0.0, 0.0], 'axis_v': [0.0, 1.0, 0.0], 'axis_w': [0.0, 0.0, 1.0], 'srs': None, 'world_id_encoding': {'enabled': True, 'column': 'world_id', 'frame': 'world_xyz', 'axis_order': ['x', 'y', 'z'], 'encoding': {'type': 'morton_zorder', 'version': '1.0', 'bits_per_axis': 24, 'axis_bits': {'x': 24, 'y': 24, 'z': 16}, 'signed': False}, 'quantization': {'scale': 10.0, 'rounding': 'nearest', 'precision_decimals': 1}, 'offset': {'x': 0.0, 'y': 0.0, 'z': 0.0, 'units': 'world'}, 'range_after_offset': {'x_min': 0.0, 'x_max': 1677721.5, 'y_min': 0.0, 'y_max': 1677721.5, 'z_min': 0.0, 'z_max': 6553.5}, 'overflow_policy': 'error', 'null_policy': 'no_nulls'}}, 'compression': {'mode': 'archive', 'default': {'codec': 'zstd', 'level': 7}, 'columns': {}}}




.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.030 seconds)


.. _sphx_glr_download_auto_examples_17_compression_framework.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: 17_compression_framework.ipynb <17_compression_framework.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: 17_compression_framework.py <17_compression_framework.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: 17_compression_framework.zip <17_compression_framework.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_