Rocstar  Software Suite

Engineering at Illinois Engineering at Illinois

Rocstar  Software Suite

Overview

Rocstar is a comprehensive, integrated, highly parallel software suite for complex three-dimensional multiphysics simulations. Its individual component codes are based in turn on basic research and development in turbulence modeling, multiphase flow, constitutive modeling, combustion chemistry, computational mechanics, coupling methodology, etc. Now relatively mature, Rocstar provides by far the most advanced rocket simulation capability available anywhere for analyzing performance and reliability issues in existing solid rocket motors and for virtual prototyping of new designs. Through collaborations with CSAR, Rocstar is increasingly relied upon by the solid propulsion programs of DOD, NASA, and the U.S. rocket industry. The code is available either by license from the University of Illinois or through a consulting arrangement with IllinoisRocstar LLC.

The original target for Rocstar was the simulation of solid propellant rocket engines, but Rocstar is in fact a flexible, general-purpose framework that can effectively simulate a wide variety of multicomponent systems involving fluid dynamics, structural dynamics, combustion, and their interactions. Rocstar features multiple, state-of-the-art solvers for various types of physical components, but what truly sets it apart is its sophisticated integration framework that enables the straightforward coupling of independently developed physics modules with minimal changes, and also provides an array of services including parallel I/O, automated load balancing, performance profiling, accurate and conservative data transfer, mesh adaptation, surface propagation, and overall parallel orchestration. Rocstar has been applied to a number of fluid-structure interaction and multiphase problems in addition to rocketry, including simulations of noise from helicopter blades, fuel injectors, volcanoes, and the effects of wind on tall buildings. In addition, Rocstar has exhibited excellent scalability on multiple thousands of processors.

Detailed Description

Rocstar is a fully-integrated, general-purpose solver for the numerical simulation of fully-coupled, time-dependent fluid, structure, and combustion interaction problems. It consists of a suite of physics applications coupled together by means of a powerful integration framework. All components of Rocstar are designed to run efficiently on massively parallel computers, enabling the use of detailed, science-based physical models in complex 3-D geometries.

The physics applications (“solvers”) integrated in Rocstar are general-purpose, discipline-specific code modules that interact with the integration framework by calling a comprehensive set of library routines to register and share their data and to take advantage of the advanced meshing, data transfer, surface propagation, I/O, and other capabilities included in the framework. A key design goal was to minimize the number of modifications required to integrate a new solver with Rocstar.

The fluid and structural dynamics applications integrated in Rocstar solve partial differential equations on moving, body-fitted computational meshes. While this choice can provide maximum solution accuracy near domain boundaries, maintaining acceptable mesh quality despite significant geometrical changes has required the development of robust and efficient mesh modification capabilities. For physical problems for which boundary fitted meshes are impractical, solvers with fixed computational meshes may be used, taking full advantage of the advanced data transfer and surface propagation schemes in Rocstar.

Geometry descriptions and computational meshes are generated off-line using commercial packages plus preprocessors specific to each solver. The preprocessors are invoked by a comprehensive Rocstar input preparation tool that largely automates problem set-up.

For further information, see the Rocstar documentation.
Fluid Dynamics

Rocstar includes complementary cell-centered, finite-volume compressible flow solvers that share a substantial portion of their code base. The fluid equations are formulated on moving meshes (ALE) to handle geometrical changes without the loss of accuracy typically incurred transferring the solution to new meshes. Both fluid solvers are integrated with additional physics modules for simulating turbulent and multiphase fluid flows. Non-ideal gases, chemical species, droplets, smoke, and radiation (flux-limited diffusion approximation) can all be included with full, two-way coupling. In addition, an advanced finite difference fluid solver based on overlapping meshes is also available

Rocflo uses either a centered or an upwind scheme with Roe flux splitting on multi-block structured meshes. In addition to explicit time stepping, Rocflo also provides a dual time stepping algorithm to enable taking longer time steps than the Courant (CFL) limit.

Rocflu operates on unstructured tetrahedral or mixed element meshes. It employs a novel high-order WENO-like approach, as well as the HLLC scheme to handle strong transient flows including shocks. Time integration is by explicit multistage Runge-Kutta or by a Newton-Krylov-based implicit scheme that utilizes the PETSc library.

Rocflo-CM is a flexible compressible flow solver that utilizes curvilinear, structured, overlapping meshes and high-order finite difference methods. Complex geometrical objects are de-scribed by a set of body-fitted meshes that may overlap. The code supports both explicit and implicit (e.g., Pade) finite differences and arbitrarily deforming meshes. In addition, the code al-lows for the adaptive use of WENO-like schemes and incorporates a hyperviscosity-based shock capturing scheme. The filtered Navier-Stokes equations may be solved for large-eddy simulations. Various explicit Runge-Kutta schemes are available for time advancement.

Rocturb provides three classes of turbulence models, including large eddy simulation (LES), Reynolds averaged Navier-Stokes (RANS), and hybrid models (ether LES with a near-wall model or detached eddy simulation). Any combination of these turbulence models can be used in different regions of a single simulation, and various subgrid scale models and wall-layer models can be selected depending upon the flows being considered.

Rocpart treats burning aluminum droplets by tracking Lagrangian superparticles, each representing many droplets, while smoke particles are evolved using our novel equilibrium Eulerian method implemented in Rocsmoke. Additional modules, Rocspecies and Rocrad, provide for chemical species and radiation.

Structural Dynamics

Two finite-element structural dynamics solvers are available in Rocstar, both of which use an ALE formulation to account for the conversion of solid propellant into the gas phase, can solve three-dimensional heat conduction, and include a variety of element types and constitutive models. An additional multiscale structural solver is used for more detailed simulations.

Rocfrac uses an explicit time stepping scheme and cohesive elements between ordinary volumetric elements to simulate rapidly propagating cracks. Rocfrac also features several specialized element types, as well as a variety of constitutive models including micromechanics-based material models that account for nonlinear interface debonding.

Rocsolid is a variationaly-based finite-strain structural solver that employs an implicit Newmark time integrator and several high performance sparse iterative solvers, including multigrid and stabilized biconjugate gradients (BiCGSTAB). Matrix-vector multiplies are implemented using an element-by-element framework, and a matrix-free approach is used to decrease storage and CPU time. Rocsolid can also call solvers such as hypre (LLNL) and Aztec (SNL). Rocsolid includes enhanced assumed-strain solid elements for modeling metallic components, and mixed-mode elements for nearly incompressible materials response. In particular, we use the Hu-Washizu variational principle to construct P2/P0 (10-node tetrahedral) and Q1/P0 (8-node brick) elements. Mixed P1/P1 elements based on a multiscale variational principle are also available. Rocsolid optionally utilizes several micromechanics based models to account for the evolution of the microstructure and include the effects of void formation and growth, damage (dewetting), and strain hardening. An objective integration algorithm is used for rate formulations in order to eliminate the generation of spurious stresses in rigid body motions. Rocsolid optionally includes cohesive element to describe the particle-matrix decohesion in quasi-static regime, and also employs the generalized finite element method (GFEM) to capture discontinuities, such as moving fronts or cracks, without cumbersome computational geometry issues.

Rocsolid-MS is a detailed three-dimensional multiscale module that simulates the thermo-mechanical response of a solid propellant from particle packing to material failure. This module is composed of several submodules for computing detailed propellant statistics, constructing the unit cell, and performing subsequent analyses, all in fully parallel mode. Multiscale data transfer is based on a mathematically rigorous homogenization procedure. This module enables detailed, full-field studies to calibrate micromechanics-based models used in larger-scale simula-tions.

Combustion

Rocburn determines the rate of propellant deflagration as computed by various one-dimensional (normal to the surface) physical models that are temperature- and pressure-dependent. Thus, for a given model, the local burn rate is obtained independently at each cell face on the burning surface. The local burn rate is determined from either an empirical pressure-power law or by solving an unsteady heat equation in the normal direction with boundary condition given by a heat-flux look-up table precomputed using Rocfire, a detailed three-dimensional heterogeneous aluminized propellant combustion simulation code. This enables accurate reproduction of dynamic burning behavior at every cell face in a large rocket at minimal additional computational cost.

Rocfire is a three-dimensional subgrid module for simulating unsteady burning of heterogeneous composite propellants at a resolution of 1 to 10 microns with detailed physical coupling between the solid and gas phases. Phenomena taken into account include the nonplanar regression of the burning surface, unsteady heat conduction in the solid, the presence of metallic particles, and oxidizer decomposition and binder-oxidizer flames. The resulting simulations provide information on the gas-phase flame structure, burning rate characteristics, and the generation of turbulence.
Integration Framework

Roccom provides the integration interface for Rocstar to facilitate the exchange of data and functions between modules, even those written in different programming languages. By making calls to Roccom, physics modules can communicate with each other transparently and also access a variety of useful services, including mesh modification, data transfer, input/output, performance profiling, and load balancing.

Rocman is the orchestration module that controls the execution of the physics applications to perform coupled time steps. Current coupling options include the simple staggered scheme with optional predictor-corrector iterations, and the improved staggered scheme of Farhat. Advanced new coupling schemes are currently under development that promise to provide greater accuracy, stability, and efficiency. Rocman also enforces interface jump conditions (derived from conservation of mass, momentum, and energy) specific to the particular coupled problem under consideration. Individual physics solvers are not required to be customized to address any specific coupled problem.

Rocface enables physics modules for abutting domains to exchange quantities across the interface between them. By construction, the data transfer scheme exactly conserves mass, momentum, and energy across the interface, even if their respective meshes do not match. Conservation is achieved by using the common refinement of the two meshes, each subdivision of which lies entirely within a cell face in both surface meshes. Interpolation errors are minimized in the least squares sense, leading to a scheme that is several orders of magnitude more accurate than previous conservative methods. Although minimizing interpolation errors involves solving linear systems, this procedure consumes only a small fraction of the run time for a typical Rocstar simulation.

Rocprop computes the motion of a surface as it regresses due to burning. Rocprop includes a robust, accurate, and efficient new surface propagation method called face-offsetting, which is based on a generalized Huygens' principle. Unlike Eulerian methods, such as level set methods, which require a volume mesh, the face-offsetting method operates directly on a Lagrangian surface mesh. Unlike traditional Lagrangian methods, which move each vertex directly along an approximate normal or user-specified direction, the face-offsetting method propagates faces and then reconstructs vertices through an eigenvalue analysis performed locally at each vertex to resolve the normal and tangential motion of the interface simultaneously. The method rigorously maintains the smoothness and integrity of the surface as it evolves, and produces accurate physical solutions even in the presence of singularities and large curvatures.

To maintain acceptable mesh quality for dynamically changing geometries, mesh modification schemes in Rocstar are employed at two levels. Rocmop provides mesh smoothing (without changing the mesh connectivity) for unstructured meshes using the Mesquite package developed at Sandia National Laboratories. Each partition calls Mesquite concurrently, and then the positions of the vertices shared by multiple partitions are averaged. If smoothing alone cannot maintain acceptable mesh quality, then either local mesh repair or global remeshing is performed by Rocrem using the commercial meshing package Simmetrix. After remeshing, the new mesh is repartitioned and the solution is accurately transferred to the new mesh.

Supporting Infrastructure

Rocstar makes use of a number of supporting software packages that provide various services, particularly in the context of highly parallel computation. Basic support for parallelism is through the Message Passing Interface (MPI), which provides application-level communication between processors. Rocstar can use either the native implementation of MPI on a given computing platform or it can also benefit from the additional capabilities, such as dynamic load balancing, provided by Adaptive MPI (AMPI), which is based in turn on the Charm++ machine independent parallel programming system. Some modules of Rocstar also take advantage of the ParFUM parallel framework for unstructured meshes. Enhanced support for parallel I/O (input/output) is provided by Rocpanda, which is an adaptation of the Panda package.

Rocketeer  is a powerful tool for visualizing three-dimensional scientific data sets. It was developed at CSAR to analyze numerical results from rocket simulations, but it can be used for viewing many other types of 3-D data. Rocketeer handles many different types of grids (e.g., non-uniform, structured, unstructured, multiblock) on which data are defined, and many different data types (e.g., scalar, vector, or tensor fields). Rocketeer can display data from multiple files or multiple data sets from the same file in a single image, and it can perform the same graphics operations on a series of data sets automatically to produce frames for animation. The latter can be done in parallel using Voyager. Rocketeer also provides many viewing options, such as cut-aways and translucent isosurfaces, to allow a clearer view of the interior of a 3-D volume, and can display images remotely without having to transfer simulation output to a local workstation.

Rocstar Availability

Rocstar source code is distributed under the Illinois Open Source License. Queries should be directed to IllinoisRocstar LLC, which also provides commercial support for Rocstar.