DLLs in Windows are basically the equivalent of "shared objects" (.so files/libraries) on Linux. The idea is that you can store functions in a shared library rather than a static library. The reason why the idea is there is due to the fact that whatever is needed from a static library (.lib on Windows, .a on Linux) is linked into the executable/binary itself. This could mean that if you use a lot of functions from that library, you may end up with a large executable/binary as well as a large memory hog for a program.
If you link against a shared library however, that library is loaded into memory whenever it is needed after which the memory is freed (ideally). This results in smaller executable/binary as well as less memory usage. For an example, I just recently tried the D programming language again using the Digital Mars D compiler (dmd-1.018 to be exact). The minimalistic code:
resulted in a 75 KB executable on Windows. This is due to the fact that the phobos library (a single library is the power behind the D programming language...) is simply a static library (phobos.lib). No matter what optimizations I did or did not include, I could not shrink it down. The D frontend to gcc (dgcc) was even larger. Since I use gcc on Windows (via MinGW) I also use the strip utility (strip.exe on Windows) to strip unnecessary symbols from the executable. However, dmd compiles to OMF format (Microsoft once used it but dropped it in favor of COFF), which strip.exe doesn't understand. As a result, that filesize and somewhat unnecessary memory usage are simple facts that must be dealt with. I posted about the issue on their newsgroups, and the UPX packing utility was recommended. However, that results in a smaller executable at the expense of more memory usage.
Simple point: shared > static (DLL > LIB, or so > a) in a lot of cases.