Loading Datasets

Automatic Type Discovery

In most cases, Viscid can automatically detect the file type based on the filename. For these cases, loading a file is as easy as,

import os

import viscid

f = viscid.load_file(os.path.join(viscid.sample_dir, 'sample_xdmf.py_0.xdmf'))

Specifying File Type

You can manually specify the file type using the file_type keyword argument to viscid.load_file

import os

import viscid

f = viscid.load_file(os.path.join(viscid.sample_dir, 'sample_xdmf.py_0.xdmf'),
                     file_type=viscid.readers.ggcm_xdmf.GGCMFileXDMF)

You can also specify the class name as a string,

import os

import viscid

f = viscid.load_file(os.path.join(viscid.sample_dir, 'sample_xdmf.py_0.xdmf'),
                     file_type='ggcm-xdmf')

Saving Datasets

HDF5 + XDMF

Grids (or lists of Fields) can be saved to an HDF5 + XDMF pair of files. Setting complevel greater than 0 enables gzip compression.

import os

import viscid

f = viscid.load_file(os.path.join(viscid.sample_dir, 'sample_xdmf.py_0.xdmf'))

grid = f.get_grid(time=0)
viscid.save_grid('example.h5', grid, complevel=9)

# or, use ``viscid.save_fields('example.h5', [f['bx'], f['by']])``

grid2 = viscid.load_file('example.xdmf')
grid2.print_tree()

Numpy File

Grids (or lists of Fields) can be saved to Numpy’s binary data files.

import os

import viscid

f = viscid.load_file(os.path.join(viscid.sample_dir, 'sample_xdmf.py_0.xdmf'))
viscid.save_grid('example.npz', f.get_grid())

# or, use ``viscid.save_fields('example.npz', [f['bx'], f['by']])``

f2 = viscid.load_file('example.npz')
f2.print_tree()

Pandas

Datasets can be converted to / from Pandas dataframes. This enables the use of Pandas’ extensive IO tools. The drawback to using pandas is that IO is not lazy. Pandas IO should not be used if the whole file cannot comfortably fit in RAM. Note that using Pandas to write HDF5 files requires PyTables to be installed.

import os

import pandas as pd
import viscid

f = viscid.load_file(os.path.join(viscid.sample_dir, 'sample_xdmf.py_0.xdmf'))
dataframe = f.to_dataframe()
dataframe.to_hdf('example.h5', 'key', complevel=9)

f2 = viscid.from_dataframe(pd.read_hdf('example.h5'))
f2.print_tree()