A shimmering metallic wall slices through a scrubby mound on the edge of a highway in Philadelphia, like a long steel blade cutting into the earth. Halfway along its length, this silvery barrier flips up, recalling the lid of a giant laptop, forming an entrance canopy that beckons you inside. As you ascend the planted knoll, you find great furrows gouged into the ground, jagged concrete sinkholes from which the peaks of colourful sculptures protrude.

Welcome to Calder Gardens, an otherworldly place conceived by Swiss architects Herzog & de Meuron to celebrate the work of Philadelphia-born Alexander Calder, master of mobiles. It is one of the strangest cultural complexes to be built anywhere in recent years. On an unpromising site no larger than a football pitch, wedged between two highways, a beguiling sequence of spaces take visitors on a journey of discovery deep into the ground. It is part barn, part cave and part rolling meadow, compressing a whole universe of different gallery types into one compact encounter.

“I have never worked on anything like this before,” says 75-year-old Jacques Herzog, who has created numerous museums and galleries around the world, often reinventing the type each time. “There was literally no brief. I felt like an artist, waking up every morning without someone telling me what to do. Architecture is never this free.”

There might not have been a defined brief, but the client certainly had strong opinions about what they didn’t want. “I had no intention of making a museum,” says Sandy Rower, Calder’s grandson and president of the Calder Foundation, who steered the $90m project. “We wanted people to be able to sit in resonance with the work and have their own mysterious, unmediated experience. My grandfather wasn’t trying to predetermine the viewer’s reaction, so we don’t want to tell people what to think or feel.”

Inside and out … two works at Calder Gardens. Photograph: Oliver Wainwright/The Guardian The project has been a long time coming. Born in 1898, Calder was the third generation in a lineage of esteemed Philadelphia artists. His grandfather sculpted the statue of William Penn that crowns City Hall , while his father created a striking fountain of reclining river gods, marooned on a roundabout nearby. But the younger and most famous Calder left the city aged eight and – bar a big mobile in the main art museum – has never had much of a presence here, until now.

His grandson describes the project as something of a spiritual quest. Rower has referred to the complex as a hypogeum, meaning an underground temple or tomb, calling it “a sacred space for self-cultivation” – and there is a ritualistic air to the place. Visitors are taken on a theatrical journey of compression and release, ushered through dark passages, then thrust into unexpectedly airy galleries, invited to peer around corners, perch in cubbies, and explore sunken gardens to discover the work on their own terms, with not a wall text in sight. The idea is not to ask when and how these sculptures were made, or what they might mean, but to submit yourself to a pure aesthetic encounter, communing with Calder’s moving creatures in this curious submerged menagerie. So does it work?

In the depths … a black mobile and a red stabile. Photograph: Iwan Baan/Herzog & de Meuron undefined The arrival sequence is worlds away from the nearby neoclassical cultural piles that line Philadelphia’s Parkway Museum District – a misnomer for an inhospitable ribbon of highways, sliced through this part of the city in the 1960s. Once you have crossed 10 lanes of traffic, you arrive at a hilly garden (sadly ringed by an ugly chainlink fence) and ascend one of several winding paths, through what will one day grow into a lush display of perennials, laid out by Dutch celebrity plantsman Piet Oudolf, finally reaching a circular plaza at the summit.

Entering beneath the building’s sharp steel flap, you find yourself in a warm timber-lined lobby, with the feel of a domestically sized Apple Store. A bleacher staircase leads down to the first glimpse of a sunken gallery, where the contorted spidery limbs of one of Calder’s “stabiles” (what he called his static sculptures) stretch out in taut arcs. A mobile dangles above, hanging like a splatter of black paint, frozen in mid-air. The rumbling interstate highway can be briefly glimpsed through a long horizontal window, providing a reminder of the outside world that will soon be forgotten as you descend deeper into the Earth.

Things start to get stranger at a second staircase, which winds down through what feels like a lava tube, lined with bubbling black sprayed concrete with the look of rugged basalt. A circular rupture gnawed through the wall provides a window on to another Calder mobile, hanging spotlit in the stygian gloom like a ghostly white tangle of coathangers. The dark stairs lead to a light-flooded central gallery, roofed by a polished plaster ceiling, which droops in a shallow bulge overhead, like the underside of a puddle. It adds a surreal, unsettling air, as if the weight of the ground above might burst through at any moment. One of Calder’s red steel animals writhes in the middle in twisted gymnastic curves, while more mobiles hang nearby, like flocks of angular birds.

Windows on either side provide views of further contraptions. In one direction, a balancing black-and-red object stands imprisoned in a cylindrical concrete well. It is ringed by a dimly lit curved corridor gallery, where paintings and drawings hang on the raw foundation walls. A side niche presents another mobile against a seamless white backdrop, with the depthless quality of a photographic studio. Another purple-painted niche houses a sort of family altar, with works by Calder’s parents and grandfather. On the other side of the gallery, a door leads to the “vestigial garden”, where a memory of the site’s historic street pattern is summoned in the form of angular concrete forms jutting out into the courtyard, their surfaces scraped with a rough, rocky finish, as if freshly clawed from the earth. (Flood risk put paid to the possibility of using rammed earth; instead, concrete is used throughout to simulate stone and soil.)

Perfectly framed … a ghostly mobile. Photograph: Iwan Baan/Herzog & de Meuron undefined “Calder’s work was about negative space,” says Herzog, “so our process was one of excavating and carving, rather than creating positive form. We see the project as an organism with different members, each with its distinct character, triggering unlikely situations. Otherwise things are too predictable.”

There is a palpable restlessness on show throughout the complex. At times it can feel like too many ideas crammed into too small a space, a virtuosic display of Herzog & de Meuron’s greatest hits, populated by incidental Calders. It feels very much tuned to our impatient age of spectacle, and the camp, stagey quality can sometimes verge on Disneyish. You half expect to turn a corner and find an animatronic Calder bursting into a rendition of It’s a Small World, or his bones dramatically lit in a carved-out ossuary.

A bit more of the artist’s presence would help. The decision to omit captions entirely can be frustrating. Rower says he doesn’t want visitors to be staring at their phones, but without any kind of explanation of the work, you might resort to Google. “We are working on other forms of interpretation,” says Juana Berrío, director of programming. “Sometimes it will be music, or a sound bath beneath one of the sculptures, or collective singing. It’s about celebration, experimentation and real-time experience.” The loaned works will be on constant rotation, she adds, so each visit will offer something new.

I left feeling entertained, but none the wiser about Calder. Perhaps that is what he would have wanted. “Theories may be all very well for the artist himself,” he once said, “but they shouldn’t be broadcast to other people.”

There is a childlike wonder to Calder’s work, as there is to the building that Herzog & de Meuron has created to display it – even down to the loos, which wallow in a rich blue resinous soup. Kids will love exploring this labyrinthine place, just as they have stared in wonder at Calder’s mobiles for generations. As the artist once quipped: “My fanmail is enormous: everybody is under six.”