I wanted to check back in and say thanks again to SteveA for the quick feedback!
I didn't actually unscrew the screw all the way since I figured that would be bad... so I was able to re-engage the screw (to some degree anyway) and set-up the arms. With the weight of the monitors, it seems to pivot appropriately.
I haven't moved the monitors around too much yet, since I'm going to be taking it down and re-setting it up somewhere else. But it's doing great so far!
I'll let you guys know if I run into anymore hiccups.