Picture Name

Blog

How to estimate the error range of the simulation


Release time:

2021-01-12

What is the most difficult part of simulation? It's not about getting a result, nor is it about getting a very complex result, but knowing whether your calculations are correct and understanding the magnitude of the error in your results.

Our textbooks and teachers have always taught us the "correct" simulation methods, but they haven't told us how large the error in the "correct" simulation methods is.

If you don't know how to use simulation software, there are seniors to guide you. If you can't handle complex cases, there are simulation experts to help you. But who can tell you how large the error is?

Without knowing the error, you can't confidently guarantee the reliability of the simulation results, and you can't make decisions based on the simulation results. So, how useful are the results of the simulation?

To make your simulation valuable, you need to know the range of errors.
 
If you ask an expert, although they may not know the exact error, they will tell you that determining the range of simulation errors is a significant challenge, and they will give you a long task list:

Grid independence analysis, model dependency analysis, boundary influence analysis...; standard case comparison analysis, experimental and simulation comparison analysis, different software comparison analysis... .

By the time you finish these analyses, you may not even find the error range, and your time for simulation will definitely be used up.
I recommend using some simple methods to estimate the error range and to "casually" find the error range during the simulation process. You don't need to run many simulation cases just to find the error range; you just need to be observant and carefully review your simulation process.
 

1. Zero Point Deviation - Basic Error

Many cases will have a zero point, which theoretically should be 0, but the simulation will yield a small value, which reflects the size of your simulation error.

For example: In a horizontal flow, the theoretical vertical force should be 0, but your calculation shows a vertical force of 1 Newton. 1 Newton is the basic error of force, indicating that the error of this simulation is unlikely to be less than 1 Newton. The minimum error of this software, this algorithm, and this grid is approximately on the order of 1 Newton.

Knowing the basic error, you understand the limitations of this simulation. If you use this simulation for optimization design and reduce the force by 1 Newton, this optimization result is likely just an error and not worth mentioning.
 

2. Standard Point Deviation - Systematic Error

In addition to the zero point, there are standard points in the case, and the deviation of the value at the standard point from the theoretical value is the systematic error of the simulation.

For example: The theoretical pressure at a point facing the incoming flow is the stagnation pressure of the incoming flow. If you write down the pressure at this point and find it is 5% less than the stagnation pressure, you can infer that the force in this simulation is underestimated by 5%.

For example: At the moment an object is in free fall, the acceleration should be the acceleration due to gravity. If you write down the calculation result and find the acceleration is 10% greater than the acceleration due to gravity, you can infer that the acceleration at other moments is also likely to be 10% too high.

This kind of difference is often not random; it is always either too low or too high, representing the range of systematic error in this simulation.

Remembering this error range allows you to know how much your simulation results are likely to be underestimated or overestimated.
 

3. Convergence Process Deviation - Fluctuation Error

Simulation has a long calculation process, during which the data fluctuates chaotically like stock market curves. When you see these fluctuation ranges getting smaller, we say the simulation has converged, and the fluctuation range at this time is the fluctuation error of the simulation.

For example: In the first 100 steps before convergence, the speed fluctuates between 99 and 101, so the fluctuation error range of the speed is 2%.

By remembering this fluctuation error range, you can clearly state in your report that due to time constraints, the convergence of the simulation calculations is insufficient, and the potential error does not exceed 2%.
 

4. Model Differences - Model Error

A common method is to compare the calculation results of different models; the difference is the error range.

This method is unreliable. Even if you keep the grid and boundaries unchanged and only change the model, the resulting changes may not necessarily be model errors. Perhaps the differences in the model are smoothed out by the grid, or the model amplifies the influence of the boundary, resulting in differences. This method seems reasonable but often yields unreasonable error ranges.

A simple method is to look up literature. You may not find an accurate model in the literature, but you can compare these "not very accurate" models.

The differences between these "not very accurate" models represent the range of model errors.
 

5. Grid Form Differences - Grid Error

There is no need to refine the grid and check if the results change; this method of grid independence analysis may not capture the correlation of the grid.

During the simulation process, you will use different forms of grids. Even if the total number of grids remains unchanged, the changes in grid forms lead to more realistic errors.

For example: Comparing one million unstructured grids with one million structured grids, or comparing one million dense grids at the front with one million dense grids at the rear, the differences between them represent the range of grid errors.
 

6. Differences Between Presence and Absence - Maximum Error

The above methods may only uncover relatively small error ranges. What if there is a particularly large error? How can you quickly find the maximum error range?

Find the maximum error using the differences between presence and absence:

Maximum error of turbulence model < Difference between turbulence and laminar flow

Maximum error of wall grid < Difference between no-slip and slip at the wall

Maximum error of boundary < Difference between infinity and nearby boundaries

Knowing how large the maximum error can be allows you to trust the simulation results.
 

7. Random Comparison - Normal Error

In fact, estimating the error range is quite simple. In the same simulation, during the debugging process, as you make changes, you may find that certain numbers always fluctuate within a certain range; this fluctuation range is approximately the error range.

However, many newcomers to the industry are scared to death during this process. During the debugging and simulation process, they find that changing one setting or adjusting a little grid can result in data increasing or decreasing exponentially. The simulation results are inconsistent with the experiments and also inconsistent with others' simulations, with errors that are astonishingly large.

The occurrence of this situation has nothing to do with basic errors, systematic errors, fluctuation errors, model errors, grid errors, or any other errors; it is simply a mistake, and a very basic one at that.

Only when your calculation results are relatively stable and roughly the same as the experiments and others' results can you qualify to discuss errors; otherwise, you can only discuss mistakes.

Error: When you have eliminated basic mistakes and are close to data from various sources, this seemingly ambiguous difference is the error range. 

 

 

Preventing basic mistakes can ensure that your simulation results are generally usable.

There is no need to specifically analyze simulation errors; paying attention to the various differences that arise randomly during the simulation process is the true error range.
 

Errors will not ruin your design; mistakes will.

 

 

Statement: The content of the above article is organized from the internet. If there are any copyright issues, please contact us immediately.