One downside to creating components with CodeIgniter is that the backend parameters will not work like Joomla components that use the typical Joomla MVC. If Joomla components were made with CodeIgniter, a strong CRUD method would help a layman update and modify databases. One more issue which could arise is coding the css. For example, using absolute urls will work in all cases on any server. In some instances, the index() and another function within the controller could have slightly different links to the stylesheet. You can always make specific stylesheets for each view or you can use absolute urls. If absolute urls are not used, the results can vary between a WAMP machine and the live site on a Linux Server.
You can read all sorts of opinions about speed and performance regarding using CodeIgniter within Joomla, but, it does load pages rather fast. On a WAMP server, a page with 300 words and an image can freshly load in .07 seconds and faster upon a loaded cache. Personally, I would not consider the speed performance as an issue for which not to use CodeIgniter within Joomla.