The very large security lapses surrounding data that have been seen in the news could be a lot worse when everything is connected in an Internet of Things (IoT). Large corporations don't seem able to fully protect very sensitive data such as credit card information.
Even worse than the breach of sensitive and private data would be the IoT being disrupted or shutdown by malicious hackers. If the vision of the IoT is fully realized, people will rely on it to keep things like food and water coming into our houses.
An alternate view to the positive aspects of the IoT is that it is more about companies getting free data about consumers so they can better sell to them. It is not so much about convenience for the every day person but for the better manipulation of them to get them to spend money.
Another thing that the IoT could lead to is people being isolated from other people. This is already an issue being raised about online social networks and the IoT could be like social networks on steroids.
Finally, the IoT woudl great advance a world of machines. This could further erode the need for work to be done by humans. This could decouple human labor from productivity.