The interest in eating insects as food is increasing, not least for sustainability reasons. However, the acceptance among consumers remains low. The aim of this review was to identify the key factors influencing consumer perception and acceptability of insect-based foods described in recent literature. Even though sustainability factors are important, they are seldom the main reasons influencing insect consumption. Instead, a complexity of emotional factors, such as disgust and neophobia, as well as familiar tastes, textures and contexts, were found to have a major influence. In addition, exposure and positive tasting experiences have been identified as important factors for increasing acceptance. It could be concluded that there is a great complexity of different factors influencing consumers’ perceptions and acceptability of insects as food.