I've use to work with Joomla! on many occasions and use to be a big advocate, until I saw what you really should be able to do with a CMS (if you can call Joomla a CMS) and I have vowed never to use it ever again.
If all you want is to get a prepackaged theme and use the core components and modules, you'll probably get by. But as soon as you want to customize the fine grained details (e.g. pagination, permissions, etc) you will keep hitting brick walls trying to find a component or module that does what the Joomla! core should have been able to do in the first place.
The administration interface can be very confusing and cluttered with distracting and vague fields and attributes.
If you're making a small site which is just a hobby and will probably not grow or expand, Joomla will probably work for you. Otherwise I would look in to more flexible systems like Drupal, eZ Publish or django.
Remember, the foundation you lay for your website can crumble as you continue to build your site if you do not choose the right concrete (CMS).