There isn't any one general answer to the problem, or they'd be doing it on the chip. It's a collection of domain-specific techniques.

For example, when dealing with money, you can do a lot with fixed-precision rather than floating point math. If you make your fundamental unit the cent rather than the dollar, everything is really integer arithmetic. You may need to drop into floating point when doing some calculations (compound interest, for example), and you need to be very judicious about when you convert back to fixed point. But you can save yourself a lot of cycles.

Or an example that I do, cost calculations in database optimizations. There, the floating point precision simply isn't necessary, though the numbers grow larger than integers can handle. The numbers are all very approximate anyway, so you don't really need to compute 24 bits of precision. You can cut off the precision and handle it with integer arithmetic, though you need to make separate allowances for the exponents. It's a bit like trying to work with a floating point number that has only a smallish mantissa (say, 8 bits). The real work is done on the exponent, but that's already integer math.

Another example is in graphics, which uses a lot of trigonometry. Since the ultimate output of the graphical calculation is integer (screen coordinates) you can use a number of design tricks to do as much of each calculation in integer as you can. But ultimately, for precision, they put a LOT of work into making graphics chips able to do vast amounts of highly parallel floating-point operations. (So much so that they're re-using those chips for other operations, like weather modeling, where it's even harder to do integer work.)

They are already doing a fair bit of work on the chips, in fact. The chip's division algorithm is very different from what you do by hand, like SRT division:

http://www.articlesbase.com/information-technology-articles/srt-division-algorithm-1830007.html

which is actually a TRInary computation. It's very fast, but still slower than integer arithmetic.

For at least some applications, it's no longer necessary to avoid floating point operations; the computers have sped up to the point where you can just let the computer do the work, and put more energy into the coding. You certainly want to avoid premature optimization, since it takes longer and often provides no benefit. But for some applications, you need specialists in the domain and in numerical methods to see which techniques will suit the problem.

If you care about speed you will want to avoid floating point operations. One way is to use fixed point numbers. If you multiply or divide by 2 you should use bitshift operations for maximum performance. i<

Try looking up fixed point math. It is like a cross between integer and floating point. Instead of being arbitrary length after the decimal point you set it so you are working at 2 or 4 digits after the decimal point. One easy way of doing it is often to just multiply the number by 100(move the decimal point over by 2) then work everything in integer math then move the decimal point back. There are already fixed point math libraries for AVR and PIC microcontrollers as well as many others.

Enjoy,
Ray