Mari was born near Hay Springs, Nebraska, the eldest of six children born to Swiss immigrants, Jules and Mary Fehr Sandoz. Her father was a violent and domineering man, who disapproved of her writing and reading. Her childhood was spent in hard labor on the home farm, and she developed snow blindness in one eye after a day spent digging the family's cattle out of a snowdrift.

She graduated from the eighth grade at the age of 17, secretly took the rural teachers' exam, and passed. Mari taught in nearby country schools without ever attending high school. At the age of eighteen she married a neighboring rancher, Wray Macumber. The marriage was unhappy and in 1919, citing "extreme mental cruelty," Mari divorced her husband and moved to Lincoln, Nebraska.

For the next sixteen years, Mari held a variety of low-paying jobs, while writing—to almost no success—under her married name, Marie Macumber. Despite her lack of a high school diploma she managed to enroll at the University of Nebraska, thanks to a sympathetic dean. During those years, she claimed she received over a thousand rejection slips for her short stories.

In 1928, when she received word her father was dying, she visited her home and was stunned by his last request—he asked her to write his life story. She began extensive research on his life, and documented his decision to become a pioneer, his hard work chiselling out a life on the prairie, his leadership within of the pioneer community, and his friendship with the local Indians in the area. The resulting book was Old Jules. In 1929 she changed her name to Mari Sandoz.

Success did not come easily to Sandoz. In 1933, malnourished and in poor health, she moved back with her mother in the Sandhills. Every major publishing house in the United States had rejected Old Jules. Before she left Lincoln, Sandoz tossed over 70 of her manuscripts into a wash tub in her backyard and burned them.

Yet she continued to write, and began work on her next novel, Slogum House, a gritty and realistic tale about a ruthless Nebraska family. By January 1934, she returned to Lincoln, and got a job at the Nebraska State Historical Society, where she became associate editor of Nebraska History magazine.

In 1935, she received word that her revised version of Old Jules had won a non-fiction contest held by Atlantic Press. Finally, her book would be published. Before that happened, she had to fight her editor to retain the distinctive Western idiom in which she had written the book, as her publishers wanted her to standardize the English used in the book.

The book was well-received critically and commercially when it was issued, and became a Book of the Month Club selection. Some readers were shocked at her unromantic depiction of the Old West, as well as her strong language and realistic portrayal of the hardships of frontier life.

Sandoz's subsequent novels, Slogum House (1937) and Capital City (1939), brought her notoriety of a different nature: hate mail and threats. The first book was considered an attack on the character of rural Nebraskans, and Capital City was perceived as an assault on the city of Lincoln.

Sandoz moved to Denver, partly to escape the backlash, and also for better research facilities. Later, she settled in New York so she could access the research material on the West, and have proximity to her publishers.