There are two major issues I address here: water vapour is not present in equal measure throughout the atmosphere, and greenhouse gases vary in the wavelengths of infrared they absorb preferentially, and hence complement each other to some extent, rather than competing for the same photons.
Water vapour is present in the atmosphere at highly variable concentrations, up to 4% at sea level (40,000 parts per million by volume, ppmv), compared with CH4, currently at about 1.8ppmv, as compared with CO2 at 387 ppmv (methane and CO2 numbers from NOAA Earth System Research Laboratory). Water vapour content of the atmosphere varies considerably because the capacity of the atmosphere for holding water vapour is temperature-limited. The average over the whole atmosphere is 0.4% (4,000 ppmv), but over 99% of this is in the lower atmosphere (troposphere).
Some of the answer to the CO2 vs. H2O issue is that CO2 is a well-mixed gas, meaning its concentration is not temperature-dependent and given time turbulence will mix any addition equally into the atmosphere, so CO2 additions contribute to the greenhouse effect throughout the atmosphere. H2O additions on the other hand are limited both by the fact that the addition can precipitate out rapidly and H2O is limited in the volume of atmosphere it can populate. But that is not the whole picture.
Look at these three pictures, lifted from Principles of Planetary Climate by Raymond T. Pierrehumbert, from which I also derive the following explanation:
The most important thing to observe in these pictures is that the peaks are in different places. This is important because the peaks represent parts of the spectrum in which each of the three greenhouse gases prefer to absorb. We cannot do an accurate calculation of the difference between each gas based on these pictures, because each data point is averaged over 50cm-1 and therefore represents a range (hence graphing five curves for each, representing the minimum, 25th percentile, median, 75th percentile and maximum absorption coefficients for the interval graphed at each point). Nonetheless, it’s clear that at the peak near 600cm-1, CO2 is a much stronger absorber than H2O. Wavenumbers in “cm-1” are common in spectroscopy, and are simply 1/wavelength in cm.
You may be wondering why methane has no peak significantly higher than the other gases if it’s so much stronger a greenhouse gas. The reason lies in the sharp drop in effectiveness of absorbing as all molecules capable of absorbing a photon near a peak in the graph have absorbed a photon, meaning any more such photons can pass straight through. Because of methane’s relatively low concentration, it is still absorbing near a peak. If all else were equal, methane would actually absorb less per increase in concentration than CO2 because its peak is not as close to the peak of outgoing infrared (which is close for a planet of the Earth’s average temperature to the 600cm-1 peak in the CO2 curve). Notice how the scale on the vertical axis is logarithmic. The approximately straight edges of the decline from the peaks in all the graphs are the reason that increases in greenhouse gas concentrations have a logarithmic relationship to temperature increase.
There’s a lot more to it than that. The graphs I illustrate here are for 10% of the Earth’s atmospheric pressure, because they are the only ones I could find on comparable axes. However, full atmospheric pressure does not change the shape of the curves in a big way. Also, if you remember your high school physics, you may wonder why we have relatively continuous graphs, since quantum physics says a molecule can only occupy specific energy states, implying that the graph should be disjoint data points. When a photon encounters a molecule at the same time as that molecule exchanges potential energy with another molecule, the difference in energy between the photon and an allowed state of the molecule can be made up (or reduced) by an exchange of kinetic energy. This is called collision or pressure broadening.
The overall factors that go into determining the greenhouse effect and in general the overall climate of a planet are very complex; you really need to read a book like Principles of Planetary Climate. But be warned: it’s heavy going if you aren’t comfortable with calculus.