Assembly of the Gag polyprotein into new viral particles in infected cells is a crucial step in the retroviral replication cycle. Currently, little is known about the onset of assembly in the cytosol. In this paper, we analyzed the cytosolic HIV-1 Gag fraction in real time in live cells using advanced fluctuation imaging methods and thereby provide detailed insights into the complex relationship between cytosolic Gag mobility, stoichiometry, and interactions. We show that Gag diffuses as a monomer on the subsecond timescale with severely reduced mobility. Reduction of mobility is associated with basic residues in its nucleocapsid (NC) domain, whereas capsid (CA) and matrix (MA) domains do not contribute significantly. Strikingly, another diffusive Gag species was observed on the seconds timescale that oligomerized in a concentration-dependent manner. Both NC- and CA-mediated interactions strongly assist this process. Our results reveal potential nucleation steps of cytosolic Gag fractions before membrane-assisted Gag assembly.