Loading...
Connecting dreams with visual brainstorming instruction
Sun, Yasheng Li, Bohan Zhuge, Mingchen Fan, Deng-Ping Khan, Salman Khan, Fahad Shahbaz Koike, Hideki
Sun, Yasheng
Li, Bohan
Zhuge, Mingchen
Fan, Deng-Ping
Khan, Salman
Khan, Fahad Shahbaz
Koike, Hideki
Supervisor
Department
Computer Vision
Embargo End Date
Type
Journal article
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Recent breakthroughs in understanding the human brain have revealed its impressive ability to efficiently process and interpret human thoughts, opening up the possibility of intervening in brain signals. In this paper, we aim to develop a straightforward framework that uses other modalities, such as natural language, to translate the original “dreamland”. We present DreamConnect, employing a dual-stream diffusion framework to manipulate visually stimulated brain signals. By integrating an asynchronous diffusion strategy, our framework establishes an effective interface with human “dreams”, and progressively refines their final image synthesis. Through extensive experiments, we demonstrate the efficacy of our method to accurately direct human brain signals in desired directions, ultimately enabling concept manipulation through direct manipulation of the functional magnetic resonance imaging (fMRI) signals. We hope that this work will motivate the use of brain signals in human-computer interaction applications.
Citation
Y. Sun et al., “Connecting dreams with visual brainstorming instruction,” Visual Intelligence 2025 3:1, vol. 3, no. 1, pp. 1–18, Jul. 2025, doi: 10.1007/S44267-025-00081-2
Source
Visual Intelligence
Conference
Keywords
Functional magnetic resonance imaging (fMRI), Brain-to-image generation, Diffusion models, Large language model (LLM)
Subjects
Source
Publisher
Springer Nature