I tried turning the rectangular metal thing on the end either right or left with my hand but it does not yield to gentle pressure just seems stuck where it is.
Welcome.
That's a typical 1/4 turn angle stop and the handle is the lever, and off should be a 1/4 clockwise rotation.
You can slightly rotate the nut behind the lever, counter-clockwise.