Enhancing photos utilizing pure language directions has turn into a pure and expressive method to modify visible content material; but, evaluating the efficiency of such fashions stays difficult. Present analysis approaches typically depend on image-text similarity metrics like CLIP, which lack precision. On this work, we introduce a brand new benchmark designed to judge text-guided picture enhancing fashions in a extra grounded method, alongside two essential dimensions: (i) practical correctness, assessed by way of mechanically generated multiple-choice questions that confirm whether or not the supposed change was efficiently utilized; and (ii) picture content material preservation, which ensures that non-targeted areas of the picture stay visually constant utilizing an object-aware masking method and preservation scoring. The benchmark consists of over 1000 high-quality enhancing examples throughout 20 various content material classes, every annotated with detailed enhancing directions, analysis questions, and spatial object masks. We conduct a large-scale research evaluating GPT-Picture-1, the most recent flagship within the text-guided picture enhancing area, towards a number of state-of-the-art enhancing fashions, and validate our automated metrics towards human scores. Outcomes present that GPT-Picture-1 leads in instruction-following accuracy, however typically over-modifies irrelevant picture areas, highlighting a key trade-off within the present mannequin habits. GIE-Bench gives a scalable, reproducible framework for advancing extra correct analysis of text-guided picture enhancing.
