Available modules to read and write NetCDF files: PyNIO, scipy.io.netcdf, NetCDF4. Each of these modules are already installed on Carter and Conte. Simply type 'module load python' in your terminal to load Python v2.7.2.
Reading and writing netcdf files with each of these packages is very similar (particularly PyNIO and scipy.io.netcdf), differing only slightly in their syntax. NetCDF4, however, does have some different features due to the new features in netcdf version 4 (e.g. groups, unlimited dimension, etc.). Note: PyNIO supports the reading of other formats besides netcdf (e.g. grib/grib2, hdf5, shapefiles, etc.--see documentation for more details)
Import the Nio module
To open a file, we use the open_file constructor:
open_file(filepath, mode, options, history, format)
filepath: the full or relative path to data
mode:
'r' : read only
'w', 'r+', 'rw', 'a': (write, read and write, append) ->opens an existing file for writing or creates the file if it doesn't exist
'c' : creates a file for writing, but throws an error if the file already exists
Optional argument that affects how data is read/written. Must be called before open_file constructor.
history:
Optional argument. It is a user specified string that will get appended to the global "history" attribute of the file that is opened for writing.
format:
If your file name does not have a conventional file extension at the end (e.g. '.nc', '.netcdf', etc.), we can still open the file with Nio by simply specifying the format. For example, you may want to read in and visualize the raw data output from a WRF run which does not have a netcdf file extension:
file = 'wrfout_d01_2036-06-20_06:00:00'
nc = Nio.open_file(file,'r',format='nc')
Note that we are reading the file, denoted by 'r'.
Print an inventory of variables/dimensions/attributes
print nc
Now that you know the variable names within the file, we can access the content either through index notation [:] or the get_value method for scalar variable (that cannot be indexed)
example:
#dimensions
w_e = nc.dimensions['west_east']
s_e = nc.dimensions['south_north']
#variables
lat = nc.variables['XLAT'][0,:,:]
lon = nc.variables['XLONG'][0,:,:]
pcp = nc.variables['RAINNC'][:]
Accessing variable attributes
p_units = nc.variables['RAINNC'].units
print p_units
1. Open a file to write to
nc = Nio.open_file('filename_to_write.nc','w')
'w': 'c':
2. Create Dimension(s)
create_variable(name, type, dimensions)
3. Create Variable(s)
create_variable(name, type, dimensions)
Name: a string specifying what you want your variable name to be Type: Dimensions: specified as a list or tuple of dimension names specified in the order you want them to appear
Special case:
4. Assign values
Special case: assign_value()
5. Write to file and close
scipy.io.netcdfimport scipy.io.netcdf as netcdf
file = ''
1. Open a file to write to
scipy.io.netcdf.netcdf_file(filename, mode='r', mmap=None, version=1)
ncfile = netcdf.netcdf_file('filename_to_write.nc','w')
2. Create Dimension(s)
netcdf_file.createDimension(name, length)
3. Create Variable(s)
netcdf_file.createVariable(name, type, dimensions)
Name: a string specifying what you want your variable name to be
Type:
Dimensions: specified as a list or tuple of dimension names specified in the order you want them to appear
Special case: Assign a scalar value to a variable that cannot be indexed
- only need to specify name and type in this case
- netcdf_file.createVariable(name, type)
4. Assign values
Special case: netcdf_variable.assignValue(value)
5. Write to file and close
nc.flush() or nc.sync()
nc.close()
NetCDF4 expands upon the NetCDF3 common data format. Every NetCDF4 file is actually an HDF5 file (but not every HDF5 file is a netcdf file). One of the main advantages of NetCDF4 include its support for larger files, unlimited dimensions (e.g. the time dimension), something called groups, and compression ability using zlib. Using Jeff Whittaker's NetCDF4 Python module is slightly different than the two previous methods discussed, but still easy to pick up. from netCDF4 import Dataset
file = 'example_file.nc'
nc = Dataset(file,'r')
file = 'filename_to_write.nc'
ncfile = Dataset(file,'w',format='NETCDF4_CLASSIC')
#create dimensions
ncfile.createDimension('lat',numlat)
ncfile.createDimension('lon',numlon)
ncfile.createDimension('level',numnewPlevs)
ncfile.createDimension('time',numtime)
#define variables
latitude = ncfile.createVariable('lats','d',('lat',))
longitude = ncfile.createVariable('lons','d',('lon',))
tv = ncfile.createVariable('Tv','d',('time','level','lat','lon'))
plevo = ncfile.createVariable('P','d',('level',))
longitude[:] = lon
latitude[:] = lat
tv[:] = Tv
plevo[:] = newPlevs
#close ncfile
ncfile.close()
|