We present a model of cultural evolution in which an individual’s propensity to engage in social learning is affected by social learning itself. We assume that individuals observe cultural traits displayed by others and decide whether to copy them based on their overall preference for the displayed traits. Preferences, too, can be transmitted between individuals. Our results show that such cultural dynamics tends to produce conservative individuals, i.e., individuals who are reluctant to copy new traits. Openness to new information, however, can be maintained when individuals need significant time to acquire the cultural traits that make them effective cultural models. We show that a gradual enculturation of young individuals by many models and a larger cultural repertoire to be acquired are favorable circumstances for the long-term maintenance of openness in individuals and groups. Our results agree with data about lifetime personality change, showing that openness to new information decreases with age. Our results show that cultural remodeling of cultural transmission is a powerful force in cultural evolution, i.e., that cultural evolution can change its own dynamics.