Fruit quality at harvest is a complex trait, including size, overall flavor (taste and texture) and visual attractiveness (color, shape), which all depend on both genotype and environment. Many processes are underlying these traits among which, cell division and expansion, water and carbon fluxes, primary and secondary metabolism, ethylene emission…etc. In this presentation, some process based simulation models describing these processes and their interactions in the control of final fruit quality, are presented. They have been developed at different structural scales, from cell to organ, and first steps of integration have been proposed. In these models, the environment is taken into account through the determination of response laws of biophysical and biochemical parameters to environmental variables (mainly temperature, light, humidity), whereas the fruit is characterized by developmental variables (age from flowering), growth variables (biomass, water and carbon content...) and metabolic variables (respiration, sugar synthesis…). The integration of the genetic control, necessary to account for the genotype x environment (GxE) interactions, may be considered through the determination of genetic coefficients. Some approaches to link model parameters to genes or quantitative trait loci are presented and the analysis of G×E interactions through a simulation model is illustrated. Finally, perspectives concerning the integration of models and knowledge’s at the different structural scales into a virtual fruit will be proposed.