Nevertheless, the difference in compressibility between liquids and gases is huge.
Gases can be compressed with small pressures so that their volume is reduced to a small fraction of the original volume, while applying great pressures to a liquid will result in a volume only slightly smaller than the original volume.
The compressibility of gases can never be neglected, while in many practical applications of liquids it is possible to approximate them as ideal liquids, which are incompressible.
The approximation of incompressible liquids is like the approximations of inextensible wires, inextensible membranes or rigid bodies. Even if such ideal objects do not exist, such models simplify the solutions of problems and for many practical cases the solutions are close enough to the solutions that would take into account the deformations or volume changes of the bodies.
Not just a bit. All materials can be compressed quite a lot, given high enough pressure. For example, at the extreme pressures in inertially confined fusion, solid deuterium starts with 0.2 g/mL density and for a brief moment is squished to about 100 g/mL. Of course it takes huge energy density to achieve such compression, and we do not see this in everyday life.
So, whether something can or can not be regarded as incompressible simply reflects what questions we are interested in, and in what range of conditions.
The "liquids are incompressible" is first a very useful approximation for many practical problems, and second it is a conceptual thing, which comes from the introductory physics, from looking at the qualitative difference in how gasses and liquids behave when the pressure changes.
If we ignore the nuances, it is possible to say that the gas fills any volume given to it, while a liquid stays at almost the same volume for a range of pressures. That is, if we put a liquid into a larger empty box, unlike the gas it does not visibly expand. (Though it may be evaporating and turning into gas.) And if we double the pressure from one to two bars, water does not visibly compress, while air under piston is squished two-fold, assuming it is cooled to keep the temperature constant. That's where "water is incompressible compared to the air" comes from.
In some situations it can be useful to treat even gasses as having constant volume. For example in aerodynamics of subsonic flows where the pressure does not change too much from point to point, it is standard to treat the gas as an incompressible fluid. This makes the theory vastly simpler, while keeping it accurate enough in this particular regime.
Exactly like that, where the compressibility of liquids and solids does not change the essence of what we are interested in in a particular situation, the solids and liquids are considered to be incompressible, even though in other more exotic situations this would not work at all.
But can't most fluids be compressed, even if just a bit?