ELODIN : naming concepts in embedding spaces

Abstract

Despite recent advancements, the field of text-to-image synthesis still suffers from the lack of fine-grained control. Using only text, it remains challenging to deal with issues such as concept coherence and concept cohesion. A method to enhance control by generating new words that can be reused throughout multiple images is proposed. Each new word, which I call “named concept”, can be mixed and matched freely with natural language, effectively expanding human vocabulary. Just as a painter combines pre-existing shades into personalized colors according to their needs, the proposed method enables combining e.g. “yellow” and “hawk” into a single word, that is, a single named concept. The new word, when present in subsequent text prompts, results in images that consistently contain the same yellow hawk. Unlike previous contributions, our method does not replicate visuals from input data. In some cases, it can generate visual concepts in a zero-shot manner, that is, without any visual input. A set of comparisons show our method to be a significant improvement over text prompts containing only natural language. Theoretical considerations on the foundations of Deep Learning are made throughout the text and Name Learning is proposed.