In the Spring of 2022, KITP hosted the program Bridging the Gap: Accretion and Orbital Evolution in Stellar and Black Hole Binaries. This program brought together researchers across a wide range of fields in astrophysics, who were all interested in understanding the same basic physics problem applied to many different astrophysical scenarios on many different length and time scales.
One important open problem is the interaction of a binary star with a disk of material circulating within and around it. This arises in different astrophysical scenarios, such as a stellar binary system at an early age when the stars and their planets are first forming. Or the binary could be a pair of black holes accreting gas and emitting powerful outflows observable across the universe. There are, of course, differences in the details between these systems, but (we think) some of the same basic physical laws apply to both.
The researchers attending this KITP program wanted to understand different basic questions about a binary interacting with a surrounding disk. When a binary star system forms, how does the disk affect the binary's orbit? When we see bright emission from an accreting black hole, how can we tell whether it's a single black hole or a binary? And if it is a black hole binary, how does the disk affect the gravitational waves produced by the binary? How do binary stars affect the disk surrounding them, and therefore the planets that subsequently form in this disk?
COMPUTATIONAL APPROACH
A powerful way of investigating these questions is with large-scale hydrodynamical calculations. The equations of gas dynamics are integrated on the computer using a hydrodynamics code, and concrete answers to the above questions can be attained in a few idealized cases.
As one might imagine, since this is such a pivotal problem in astrophysics, many teams of researchers have simulated this on the computer. In fact, it has been investigated with at least a dozen different computational approaches and distinct codes, sometimes arriving at seemingly contradictory answers between research groups. As an example, some groups disagreed on such a fundamental question as "does the presence of the disk cause the binary's orbit to get wider or contract closer?" The fact that there was still disagreement on such a basic question demonstrates that this is a hard problem that is not easy to simulate.
Surface density for all codes at 1, 10, 100, and 300 orbits.
Credit: Duffell et al.
When everyone has their own code, and not everyone agrees on the answer, it's easy to blame disagreements on differences between codes. "Your code doesn't resolve the gas well enough." "Your code doesn't conserve angular momentum." "Your code doesn't evolve the binary self-consistently." Such discourse can be useful sometimes, but it also entrenches us in our own camps, with our own codes and our own opinions about the "correct" answer.
THE CODE COMPARISON
With that in mind, during this KITP program, a multitude of researchers across disciplines, many in frequent direct competition with one another, set aside their differences and tested all their codes' performance on a single benchmark test problem. This was known as the "Santa Barbara Binary Disk Code Comparison". A comparable project initiated at KITP was the "Santa Barbara Cluster Code Comparison". Performed in 1999, 13 different groups tested their codes' ability to simulate structure formation in the early universe. The paper describing those results (Frenk et al. 1999) has now been cited more than 400 times. The Santa Barbara Binary pulled together 16 different researchers using 11 very different numerical methods.
The phrase "code comparison" is a bit of a misnomer. It invokes the idea that we're competing to see which code is "the best". Really, the goal was to see if we can reach a consensus solution as a community, and if so, what is necessary for each code to converge to this solution? This attitude was an important component of the project, because if the different simulators thought of this as a competition, we might retreat to our camps claiming the other groups' codes were just "doing it wrong".
KITP provided an ideal neutral ground for the code comparison where everyone felt comfortable sharing their code's output data without worrying if it would be turned against them, or used to argue their code was not trustworthy. A great deal of trust is necessary for a project like this, and the thoughtful design of the KITP programming and atmosphere creates a climate of trust and mutual respect, even among fierce competitors.
The comparison was successful at establishing a benchmark solution that all codes could agree on (for example, all codes agreed that the disk torques the binary outward for the benchmark problem, and they even agreed on the strength of this torque). We anticipate this will have a very strong impact in both relevant fields of stellar and black hole binaries. We anticipate a particular impact on early-career researchers trying to break into the field and getting their codes off the ground. There is now an agreed-upon test problem with public data output from all participating codes—representing the state-of-the-art—that anyone can test their own code against.
The work was published in The Astrophysical Journal in 2024 and reads almost as an unofficial review of the field. The code comparison represents the best of what academic collaboration can be. We disagree, we debate, and we are critical of one another, but at the end of the day, we all want to understand what's really going on. Under the right conditions, like those at KITP, we can get together and honestly assess whether or not we really disagree, or if we are just asking different questions.
by Paul Duffell Professor, Purdue University