Continuous Integration (CI) testing is an expensive, time-consuming, and resource-intensive process. Test case prioritization (TCP) can effectively reduce the workload of regression testing in the CI environment, where Reinforcement Learning (RL) is adopted to prioritize test cases, since the TCP in CI testing can be formulated as a sequential decision-making problem, which can be solved by RL effectively. A useful reward function is a crucial component in the construction of the CI system and a critical factor in determining RL’s learning performance in CI testing. This paper focused on the validity of the execution history information of the test cases on the TCP performance in the existing CI testing optimization methods based on RL, and a Dynamic Time Window based reward function are proposed by using partial information dynamically for fast feedback and cost reduction. Experimental studies are carried out on six industrial datasets. The experimental results showed that using dynamic time window based reward function can significantly improve the learning efficiency of RL and the fault detection ability when comparing with the reward function based on fixed time window.