.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/08_sparse_parquet_to_pbm.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_08_sparse_parquet_to_pbm.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_08_sparse_parquet_to_pbm.py:


Dense and Sparse PBM from XYZ Parquet
=====================================

This example shows an axis-aligned workflow that starts dense and then creates
a sparse model from filtered rows.

Workflow:

1. Create a source Parquet file with a synthetic ``fe`` column.
2. Create a dense PBM via :meth:`~parq_blockmodel.ParquetBlockModel.from_parquet`.
3. Filter the source rows to ``fe > 57``.
4. Create a second PBM from that sparse input.
5. Compare dense and sparse models visually.

.. GENERATED FROM PYTHON SOURCE LINES 17-29

.. code-block:: Python


    import tempfile
    from pathlib import Path

    import pandas as pd
    import pyarrow as pa
    import pyarrow.parquet as pq
    import pyvista as pv

    from parq_blockmodel import ParquetBlockModel
    from parq_blockmodel.utils.demo_block_model import create_toy_blockmodel








.. GENERATED FROM PYTHON SOURCE LINES 30-34

Create a Centroid Parquet File
------------------------------
Use the toy helper to write a parquet file containing ``x``, ``y``, ``z``
centroid columns and a synthetic ``fe`` grade.

.. GENERATED FROM PYTHON SOURCE LINES 34-65

.. code-block:: Python


    temp_dir = Path(tempfile.gettempdir()) / "sparse_parquet_to_pbm"
    temp_dir.mkdir(parents=True, exist_ok=True)

    source_parquet = temp_dir / "dense_source.parquet"
    toy_df = create_toy_blockmodel(
        shape=(14, 12, 8),
        block_size=(10.0, 10.0, 5.0),
        corner=(0.0, 0.0, 0.0),
        grade_name="fe",
        grade_min=45.0,
        grade_max=70.0,
        deposit_center=(70.0, 60.0, 20.0),
        deposit_radii=(45.0, 35.0, 18.0),
        noise_std=0.0,
    )

    # Build an xyz-indexed source parquet for the external-use-case scenario.
    source_df = toy_df.reset_index(drop=False)
    source_df = source_df.drop(columns=["i", "j", "k"], errors="ignore")
    source_df = source_df.set_index(["x", "y", "z"]).sort_index()
    source_df.to_parquet(source_parquet)

    loaded_source_df = pd.read_parquet(source_parquet)
    if loaded_source_df.index.names != ["x", "y", "z"]:
        raise RuntimeError("Expected source parquet to load with MultiIndex ['x', 'y', 'z'].")
    if any(col in loaded_source_df.columns for col in ["i", "j", "k"]):
        raise RuntimeError("Source parquet must not contain i, j, k columns.")

    loaded_source_df.head()






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th></th>
          <th></th>
          <th>block_id</th>
          <th>depth</th>
          <th>depth_category</th>
          <th>fe</th>
        </tr>
        <tr>
          <th>x</th>
          <th>y</th>
          <th>z</th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th rowspan="5" valign="top">5.0</th>
          <th rowspan="5" valign="top">5.0</th>
          <th>2.5</th>
          <td>0</td>
          <td>37.5</td>
          <td>deep</td>
          <td>45.0</td>
        </tr>
        <tr>
          <th>7.5</th>
          <td>1</td>
          <td>32.5</td>
          <td>deep</td>
          <td>45.0</td>
        </tr>
        <tr>
          <th>12.5</th>
          <td>2</td>
          <td>27.5</td>
          <td>deep</td>
          <td>45.0</td>
        </tr>
        <tr>
          <th>17.5</th>
          <td>3</td>
          <td>22.5</td>
          <td>deep</td>
          <td>45.0</td>
        </tr>
        <tr>
          <th>22.5</th>
          <td>4</td>
          <td>17.5</td>
          <td>shallow</td>
          <td>45.0</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 66-69

Create Dense PBM
----------------
Promote the raw centroid parquet to the canonical ``.pbm`` format.

.. GENERATED FROM PYTHON SOURCE LINES 69-73

.. code-block:: Python


    pbm_dense = ParquetBlockModel.from_parquet(source_parquet)
    pbm_dense





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    ParquetBlockModel(name=dense_source, path=/tmp/sparse_parquet_to_pbm/dense_source.pbm)



.. GENERATED FROM PYTHON SOURCE LINES 74-78

Filter to High-Grade Rows (fe > 57)
-----------------------------------
Filter the original block rows, then write a sparse parquet while preserving
geometry metadata from the dense PBM so the sparse model keeps the same grid.

.. GENERATED FROM PYTHON SOURCE LINES 78-90

.. code-block:: Python


    source_table = pq.read_table(pbm_dense.blockmodel_path)
    source_df = pd.read_parquet(source_parquet)

    filtered_df = source_df[source_df["fe"] > 57.0].copy()
    filtered_df = filtered_df.reset_index().sort_values(["x", "y", "z"])

    if filtered_df.empty:
        raise RuntimeError("No rows matched fe > 57. Lower the threshold or adjust toy model parameters.")

    filtered_df.head()






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>x</th>
          <th>y</th>
          <th>z</th>
          <th>block_id</th>
          <th>depth</th>
          <th>depth_category</th>
          <th>fe</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>55.0</td>
          <td>55.0</td>
          <td>17.5</td>
          <td>523</td>
          <td>22.5</td>
          <td>deep</td>
          <td>58.501200</td>
        </tr>
        <tr>
          <th>1</th>
          <td>55.0</td>
          <td>55.0</td>
          <td>22.5</td>
          <td>524</td>
          <td>17.5</td>
          <td>shallow</td>
          <td>59.853246</td>
        </tr>
        <tr>
          <th>2</th>
          <td>55.0</td>
          <td>55.0</td>
          <td>27.5</td>
          <td>525</td>
          <td>12.5</td>
          <td>shallow</td>
          <td>58.153335</td>
        </tr>
        <tr>
          <th>3</th>
          <td>55.0</td>
          <td>65.0</td>
          <td>12.5</td>
          <td>530</td>
          <td>27.5</td>
          <td>deep</td>
          <td>59.669567</td>
        </tr>
        <tr>
          <th>4</th>
          <td>55.0</td>
          <td>65.0</td>
          <td>17.5</td>
          <td>531</td>
          <td>22.5</td>
          <td>deep</td>
          <td>60.954389</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 91-99

.. code-block:: Python

    sparse_parquet = temp_dir / "sparse_fe_gt57.parquet"
    sparse_table = pa.table({col: filtered_df[col].to_numpy() for col in filtered_df.columns})
    sparse_table = sparse_table.replace_schema_metadata(source_table.schema.metadata)
    pq.write_table(sparse_table, sparse_parquet)

    print(f"Dense rows : {len(source_df):,}")
    print(f"Sparse rows: {len(filtered_df):,}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Dense rows : 1,344
    Sparse rows: 34




.. GENERATED FROM PYTHON SOURCE LINES 100-103

Create Sparse PBM
-----------------
The sparse parquet is now promoted to a second PBM.

.. GENERATED FROM PYTHON SOURCE LINES 103-110

.. code-block:: Python


    pbm_sparse = ParquetBlockModel.from_parquet(sparse_parquet)

    print(f"Dense is sparse?  {pbm_dense.is_sparse}")
    print(f"Sparse is sparse? {pbm_sparse.is_sparse}")
    print(f"Sparse fraction   {pbm_sparse.sparsity:.2%}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Dense is sparse?  False
    Sparse is sparse? True
    Sparse fraction   97.47%




.. GENERATED FROM PYTHON SOURCE LINES 111-114

Compare the Two Models
----------------------
Visualize both models side-by-side with a shared color range.

.. GENERATED FROM PYTHON SOURCE LINES 114-138

.. code-block:: Python


    dense_grid = pbm_dense.to_pyvista(grid_type="image", attributes=["fe"])
    sparse_grid = pbm_sparse.to_pyvista(grid_type="unstructured", attributes=["fe"])

    fe_min = float(source_df["fe"].min())
    fe_max = float(source_df["fe"].max())

    plotter = pv.Plotter(shape=(1, 2))

    plotter.subplot(0, 0)
    plotter.add_text("Dense PBM", font_size=11)
    plotter.add_mesh(dense_grid, scalars="fe", clim=[fe_min, fe_max], show_edges=False)
    plotter.show_axes()

    plotter.subplot(0, 1)
    plotter.add_text("Sparse PBM (fe > 57)", font_size=11)
    plotter.add_mesh(sparse_grid, scalars="fe", clim=[fe_min, fe_max], show_edges=False)
    plotter.show_axes()

    plotter.link_views()
    plotter.show()






.. image-sg:: /auto_examples/images/sphx_glr_08_sparse_parquet_to_pbm_001.png
   :alt: 08 sparse parquet to pbm
   :srcset: /auto_examples/images/sphx_glr_08_sparse_parquet_to_pbm_001.png
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.743 seconds)


.. _sphx_glr_download_auto_examples_08_sparse_parquet_to_pbm.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: 08_sparse_parquet_to_pbm.ipynb <08_sparse_parquet_to_pbm.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: 08_sparse_parquet_to_pbm.py <08_sparse_parquet_to_pbm.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: 08_sparse_parquet_to_pbm.zip <08_sparse_parquet_to_pbm.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_