Based on the simple what first comes to mind rule, the theory of visual attention (TVA; Bundesen, 1990) provides a comprehensive account of visual attention that has been successful in explaining performance in visual categorization for a variety of attention tasks. If the stimuli to be categorized are mutually confusable, a response rule based on the amount of evidence collected over a longer time seems more appropriate. In this paper, we extend the idea of a simple race to continuous sampling of evidence in favor of a certain response category. The resulting Poisson random walk model is a TVA-based response time model in which categories are reported based on the amount of evidence obtained. We demonstrate that the model provides an excellent account for response time distributions obtained in speeded visual categorization tasks. The model is mathematically tractable, and its parameters are well founded and easily interpretable. We also provide an extension of the Poisson random walk to any number of response alternatives. We tested the model in experiments with speeded and nonspeeded binary responses and a speeded response task with multiple report categories. The Poisson random walk model agreed very well with the data. A thorough investigation of processing rates revealed that the perceptual categorizations described by the Poisson random walk were the same as those obtained from TVA. The Poisson random walk model could therefore provide a unifying account of attention and response times.