This shows you the differences between two versions of the page.

Both sides previous revision Previous revision | Last revision Both sides next revision | ||

kspdiverged [2014/07/11 20:28] Ed Bueler |
kspdiverged [2014/07/11 20:58] Ed Bueler clarified that it can be either of the "effective viscosity not converged" or "KSPSolve() ... diverged" cases, and integrated Jed's comment in reasonable way |
||
---|---|---|---|

Line 19: | Line 19: | ||

</code> | </code> | ||

- | This is a sign that the problem seen by the elliptic stress balance solver, the SSA solver, is for some reason too difficult. Roughly speaking, the condition number of the system is large, although it may be that your options for PISM have generated an actually unsolvable system. | + | This message might also be like |

+ | <code> | ||

+ | PISM WARNING: KSPSolve() reports 'diverged'; reason = -3 = 'DIVERGED_ITS' | ||

+ | writing linear system to PETSc binary file SSAFD_kspdivergederror.petsc ... | ||

+ | </code> | ||

+ | | ||

+ | These failures are signs that the problem seen by the elliptic stress balance solver, the SSA solver, is for some reason too difficult. Roughly speaking, for the first case the computed estimate of the temperature-dependent strength of the shear-thinning ice is not behaving well, while for the second case, the condition number of the system is large, although it may be that your options for PISM have generated an actually unsolvable system. | ||

Why do these errors occur sporadically? As far as we understand, the reason is that at some timestep and some location in the ice sheet there is very thin ice (e.g. < 10 m), or there is ice with very weak basal resistance. At the same places we think there is high driving stress or membrane stress gradients. That is, the problem is a local nastyness for the stress balance, generating one or a few rows of the linear system that have nearly-zero coefficients. (//If the reason were really clear we would probably have already fixed it. Simple and quickly-reproducible cases that generate these errors, on coarse grids, would be greatly appreciated.//) | Why do these errors occur sporadically? As far as we understand, the reason is that at some timestep and some location in the ice sheet there is very thin ice (e.g. < 10 m), or there is ice with very weak basal resistance. At the same places we think there is high driving stress or membrane stress gradients. That is, the problem is a local nastyness for the stress balance, generating one or a few rows of the linear system that have nearly-zero coefficients. (//If the reason were really clear we would probably have already fixed it. Simple and quickly-reproducible cases that generate these errors, on coarse grids, would be greatly appreciated.//) | ||

Line 25: | Line 31: | ||

So, how do you resolve these errors in PISM runs? | So, how do you resolve these errors in PISM runs? | ||

- | - The above example was probably an option failure, where an iceberg was created for which the stress balance is ill-posed (i.e. not invertible even in theory). You should use ''-kill_icebergs'' or ''-pik'' if the above occurs. | + | - The failure may be resolved by adding options. Specifically, an iceberg may have been created, for which the stress balance is ill-posed (i.e. not invertible even in theory). You should use ''-kill_icebergs''. More general advice is to use ''-pik'', which implies ''-kill_icebergs'', for most runs. |

- | - In some cases you may observe that the system is solvable with the above stronger options, something like <code> | + | - The problem occurs most often with steep bed topography. The special problematic case may be nunataks, i.e. steep mountains poking up through the ice sheet. Consider using a smoother interpolation scheme for the bed topography, if that is justified given the bed elevation data you have. |

- | $ ./ex10 -f SSAFD_kspdivergederror.petsc -ksp_type gmres -ksp_norm_type unpreconditioned -ksp_pc_side right -pc_type asm -sub_pc_type lu | + | - In the "''KSPSolve() reports 'diverged'''" case you may observe that the system is solvable with stronger PETSc linear solver options. Specifically, you can add these "direct subdomain solves" options to your PISM run: <code> |

- | Number of iterations = 3 | + | |

- | Residual norm 431.332 | + | |

- | </code> In this case you can add these "direct subdomain solves" options to your PISM run: <code> | + | |

-ksp_type gmres -ksp_norm_type unpreconditioned -ksp_pc_side right -pc_type asm -sub_pc_type lu | -ksp_type gmres -ksp_norm_type unpreconditioned -ksp_pc_side right -pc_type asm -sub_pc_type lu | ||

</code> The run will likely be about a factor of two slower but it will probably not generate the errors. For more discussion of PETSc solver options which may help with performance and robustness of PISM runs, see [[petscoptions|the page on PETSc solver options]]. | </code> The run will likely be about a factor of two slower but it will probably not generate the errors. For more discussion of PETSc solver options which may help with performance and robustness of PISM runs, see [[petscoptions|the page on PETSc solver options]]. | ||

- | - Finally, the problem occurs most often with steep bed topography, and especially nunataks (i.e. steep mountains poking up through the ice sheet). Consider using smoother interpolation schemes for the bed topography, if that is justified given the actual bed elevation data you have. | ||

- | FIXME: | + | |

- | Jed Brown, [[http://www.mcs.anl.gov/petsc/petsc-2/miscellaneous/index.html|a PETSc developer]] and an original PISM author, has the following diagnostic advice: //You can load in the binary file and use solvers on that single linear system. This scales to large problems (e.g. SSA solves on 10km Antarctica grids and such), and is not just a toy. Here is testing using ''ex10'' which is the compiled form of ''src/ksp/ksp/examples/tutorials/ex10.c'' from the PETSc source code. We try direct subdomain solves, a robust starting point to see that the system is not invertible in this case:// | + | === Examining a saved linear system in a .petsc file === |

+ | | ||

+ | This comment applies in the "''KSPSolve() reports 'diverged'''" case. | ||

+ | |||

+ | Jed Brown, [[http://www.mcs.anl.gov/petsc/petsc-2/miscellaneous/index.html|a PETSc developer]] and a PISM author, has the following diagnostic advice: //You can load in the binary file and use solvers on that single linear system. This scales to large problems (e.g. SSA solves on 10km Antarctica grids and such), and is not just a toy. Here is testing using ''ex10'' which is the compiled form of ''src/ksp/ksp/examples/tutorials/ex10.c'' from the PETSc source code. We try direct subdomain solves, a robust starting point to see that the system is not invertible in this case:// | ||

<code> | <code> | ||

$ ./ex10 -f SSAFD_kspdivergederror.petsc -ksp_type gmres -ksp_norm_type unpreconditioned -ksp_pc_side right -pc_type asm -sub_pc_type lu | $ ./ex10 -f SSAFD_kspdivergederror.petsc -ksp_type gmres -ksp_norm_type unpreconditioned -ksp_pc_side right -pc_type asm -sub_pc_type lu |

kspdiverged.txt · Last modified: 2014/10/11 00:36 by Constantine Khroulev