The Robinson family on their way to New Guinea is chased by pirates into a storm. The captain and crew abandon the ship leaving the family shipwrecked off an uninhabited island. Father (John Mills) and his two eldest teenage sons Fritz and Ernst (James MacArthur and Tommy Kirk) salvage as much as they can from the wreck including livestock, tools, and even an organ. As they gather what can be removed from the ship, the pirates return and begin shooting at the ship. Fritz and Ernst begin readying the ship's cannon, but they only have one shot. Suddenly, the pirates turn around; Father has put up a flag indicating the ship is under quarantine and that there is Black Death aboard. The three men construct a tree house home on the island while the youngest boy Francis (Kevin Corcoran) investigates the wildlife and starts an impressive collection of animals including a young elephant, a capuchin monkey and two Great Danes rescued from the ship which they name Duke and Turk.
Mother (Dorothy McGuire) prays to be rescued. The boys, particularly Ernst, also build inventions to provide modern amenities to the family such as drawing water and preserving food. Fritz and Ernst head off to explore the island to see if anyone else lives on it, or if it is connected to any other lands. While at a distant corner of the island, they again spot the pirates who originally drove them into the storm. The pirates have captured another ship and have the ship's captain and teenage cabin boy "Bertie" captives for ransom. Fritz and Ernst rescue Bertie but the captain, Bertie's grandfather, stays behind.
The three dodge the pirates and head back to the family's tree house. En route, they discover that Bertie is really Roberta (Janet Munro) and not a boy at all. She tells them that her grandfather did not want the pirates to know that she was a girl out of fear to what the pirates would do to her. The three fend off snakes and hyenas as they head home and even rescue a zebra foal, then lead it home.
They arrive back at the treehouse at Christmas. Father realizes the pirates will try to reclaim Roberta and decides to make a stand against them. Defenses are prepared by building booby-traps and fortifying a rocky outcropping. They blow up the ship's wreckage in an attempt to make it difficult for the pirates to remember where the family went aground. While prepping for the pirate attack, Fritz and Ernst vie for the affections of Roberta. Ernst is more studious and attempts to impress her with his knowledge and intelligence while Fritz, older and more experienced, uses his charm and physical attributes to attract her. Fritz and Ernst eventually come to blows over her and are stopped only by the intervention of Father. He declares the next day the first holiday for "New Switzerland" in an attempt to divert everyone.
While prepping for the race (everyone has an animal to ride: Francis has the elephant, Ernst the ostrich, Roberta the zebra, the monkey on Duke, etc. ), Mother fires the gun to signal the start of the race; the pirates, who are at that time sailing the coast looking for the place they last saw the ship that brought the family to the island, hear the gun and know the family is near. Led by their captain (Sessue Hayakawa), the pirates storm the island. The family manages a brave defense but are sorely pressed.
Their defenses include pits with a tiger in one pit, rock piles, a log pile and coconut bombs (hollowed out coconuts filled with gunpowder with a fuse), all of which cause problems for the attacking pirates who suffer many casualties. When the pirate leader waves a white flag the family imagines they have routed the pirates, but a group of pirate reinforcements instead are sneaking around the back of the fort and slowly scailing the rockface cliff from the shore leading up to the Robinson family strongpoint. Francis' much-maligned "pirate alarm" (a series of bells stringed together on a tree branch) is the only thing that warns them of the surprise attack. They begin defending the fortress, but are soon down to only a few shots with their muskets. At this critical moment when all seems lost, a ship appears on the horizon captained by Roberta's grandfather (Cecil Parker) and fires its cannons while the retreating pirates are trying to make a desperate escape. The cannon fire hits the pirate ship, which sinks while the family rejoices. However, Father, Mother and Francis choose to remain on the island with Duke, Turk, and Francis' collection of animals while the Captain notes that Father will likely be recommended as Governor of the new colony. As for the rest of the family, Ernst chooses to return to Europe with the rescuers in order to enroll in a university to continue his studies while Fritz and Roberta plan stay behind on the Robinson island to marry and make the island, New Switzerland, their home.