It's great that you're cautious, and it does indeed seem that your caution has paid off. I'm cautious as well, and I haven't had a virus in at least 15 years, that I can remember. The last thing I got was an Excel / Word Macro virus back in the mid 1990s. But I have always kept anti-virus running, just in case.
Even the few web sites that you specifically allow can be hacked, and infected with trojans, worms, and other malware. If you're on the internet at all, it doesn't hurt to have anti-virus software running. Viruses don't come just through web sites; they can come through emails, or files people send to you in email. If you use Instant Messaging, that is one of the highest security risks around; a lot of companies block IM entirely for that reasons.
Anti-virus software is like insurance; if you're careful, most of the time you won't need it, but you always keep it around for the one time that you do need it. Using Firefox and restricted accounts is helpful, but not guaranteed to completely eliminate viruses, especially as they become more sophisticated all the time. And you don't actually have to be browsing to get infected; hackers can break through firewalls and install botnets or other malware onto your machine even when you're not using the machine. As long as it's up and running and connected, people and robots are out there trying to hack into it.
And I should mention that anti-virus software isn't enough anymore. In fact, viruses are my *least* concern. I'm much more concerned about other types of malware -- particularly botnets that take over your machine and become part of the network of spammers. In addition to anti-virus software, you have a firewall running (if you don't already), and apply security patches to your system regularly. And it's helpful to run more than one anti-malware program, since none of them are 100% accurate. Anti-virus software today has more malware capability, but things like Lavasoft's Ad-Aware and the "Microsoft’s Malicious Software Removal Tool" can help find things that Norton, Avast, McAfee, and others either don't, or cannot.
Here is an interesting article on the subject: www.nytimes.com/2008/10/21/technology/internet/21botnet.html. One of the most enlightening statements in the article describes what happens when an unprotected computer (no firewall) is connected to the internet -- “The mean time to infection is less than five minutes . . .” Firewalls are designed to prevent that sort of problem, but no protection scheme is perfect, short of disconnecting from the internet entirely.
And finally, I would suggest that, just because Avast didn't find anything is no guarantee that you don't have a malware infection. Many of today's sophisticated malware (botnets included) can prevent anti-malware software from seeing them *if* the infection takes place before the anti-malware software was loaded. So even if Avast doesn't seen any infections on your system today, it could prevent future issues. What's more, some malware is being written not only to hide itself from anti-malware programs, but to remove competing malware from the system. Often malware is detected because the computer starts running slower, or stuff clashes due to conflicts between various malware. If the malware can prevent other malware from getting on the system, it can minimize the risk of infection. Here's an article describing one instance where a Trojan installed it's own anti-virus program: http://www.eweek.com/c/a/Security/Spam-Trojan-Installs-Own-AntiVirus-Scanner.
So, I would say, if even the hackers who want to infect your system think they need protection from malware, how much more important is it to protect yourself from these very intelligent and malicious malware creators?