Fascia board install - unlevel rafter tails

Hi,

I'm a first timer in the process of replacing fascia board. House has a hip roof with 2x4 rafters square cut to 2x4 fascia board. First 18" of plywood is taken off all the way around. I took the old fascia board off and ran into an issue - rafter ends are not level horizontally (up to 1/2"). I think it's mostly due to shims (between top of the wall and rafter) not being of a consistent thickness as well as several rafters having some axial twisting. Here is a sketch:

https://docs.google.com/document/d/1zGd3xBP678jb3D6Kvl3Z9chBDlrBDSiRH7t1bgKMtds/edit?hl=en_US&pli=1


My question is how to choose the best level to start from? My first thought was to put up a first 2x4, try to average out how much rafter ends are not flush with the top of the 2x4 with the same amount of other (lower rafters) below the bottom and find a median. With this in mind I installed first board at the corner, but after turning the corner and nailing 2nd board level with the first one, I ended with all rafter ends (that 2nd board is sitting against) higher that the board, i.e. my original "centerline" was chosen pretty low. I was wondering if there is a better way of doing this? Should I run a string line around the whole house to get a better level idea of all rafter tails? Should I base my level of the lowest rafter end and let plywood follow the rafters as they are instead of installing fascia board higher than some low rafter ends and forcing plywood end to sit of the fascia board with a gap to those rafters? That way I will have some gaps between plywood an 2x4 fascia, which will be covered by drip edge anyways.
Pardon my long write up.
Your input is greatly appreciated.

You're fighting an uphill battle if your intent is to install level fascia boards on a house having rafter ends that may not be close to level. Have you shot a level (laser or string) line all the way around, to determine how much variation there is? I'd be inclined to match the median of what's there, shimming and trimming any individual differences. When you're all done and things are buttoned up, only you will know.

We see this kind of stuff on new trusses all the time, I would use 2x6 facia, it hides the defects better. Install it level with the highest rafters and scab 18" 2x4s on the side of the low ones to bring the plywood level at the edge.