Yes, fillers and, you are essentially correct. Small powder charges lay horizontal in the case and the primer "flashes over" the charge causing poor ignition. By using a filler the powder is held against the primer and ignition and performance is much improved.. However, they aren't to be used will-nilly. It depends on what kind of powder, what case/cartridge and the firearm it's to be used in. They're hardly ever used in handgun cartridges. I've been using fillers since the late 70's or early 80's years and have never had an unsafe condition arise or blown up a firearm or even had a bad load. In times past all kinds of materials were used. During the BP days the British were famous for using cork wads and some of that carried over to smokeless loads. Kapok was used in the late 1800's and 1900's through probably the 50's. Cream of Wheat was popular in the 40's and through the 70's and into the 80's even. About then Polyester/Dacron gained popularity and remains so. Since the 80's lots of pressure work has been done regarding fillers. Solid fillers, cork and the like as well as cereals have mostly been abandoned because of excessive pressure excursions. Dacron has proven safe as has open cell foam and cotton balls when fluffed up.
Lyman's 47th loading manual was still recommending Dacron for fillers and they had been recommending them for 75 or more years. The only thing wrong with fillers is the same thing "wrong" with a lot of things. People use them carelessly or ignorantly then blame the object for their stupidity. Much like the firearm is the cause of crime. That and corporate lawyers shielding their clients from liability....which today translates into a near complete lack of personal responsibility and a sue happy population.