Chosen Solution

How do I find and change this little battery?

If you look at the repair guide on iFixit for removing the logic board Here: iMac G3 Model M4984 Logic Board Replacement the battery is on the top of the logic board once you remove the assembly, there might be a black cover over it, but it uses a 3.6V 1/2 AA ( Lithium ) you ca buy them at radioshack (And pay an arm and a leg), or online where they are much cheaper