After installing the new compilation environment, you can clean up the remains of the old environment.
You can remove the old library files. Look in /lib for files named ld-linux.so.1.?.?. You can delete them all except the one with the highest version number (ld-linux.so.1 should be linked to it). You can do the same for libdl.so.1.?.? in /usr/lib.
You can remove the old GCC target and version files. Look in /usr/lib/gcc-lib. There should be a directory for each target. Unless you have installed other targets, you can delete each directory except for i486-linux-libc5 and i486-linux-libc6.
Under each target directory should be exactly one directory named after the GCC version. You can delete all other directories.
You can remove the old library files. Look in /lib for files named libc.so-5.?.?. You can delete them all except the one with the highest version number (libc.so.5 should be linked to it; if not, try ldconfig -v and check again). You can do the same for libm.so-5.?.?.
Be careful with removing old library files. As the main version number changes between releases, old libraries are still linked to the old libraries and will stop working if you delete them. You should first check with ldd whether there are any binaries on your system linked to them.
You can remove any old target directories in /usr that are still present (and unused). A common one is /usr/i486-linux.