Stability of a One-degree-of-freedom Canonical System in the Case of Zero Quadratic and Cubic Part of a Hamiltonian

We consider the stability of the equilibrium position of a periodic Hamiltonian system with one degree of freedom. It is supposed that the series expansion of the Hamiltonian function, in a small neighborhood of the equilibrium position, does not include terms of second and third degree. Moreover, we focus on a degenerate case, when fourth-degree terms in the Hamiltonian function are not enough to obtain rigorous conclusions on stability or instability. A complete study of the equilibrium stability in the above degenerate case is performed, giving sufficient conditions for instability and stability in the sense of Lyapunov. The above conditions are expressed in the form of inequalities with respect to the coefficients of the Hamiltonian function, normalized up to sixth-degree terms inclusive.