Yep, this is the perfect time to make changes.
You can't just remove the receptacle and cover it up.
Especially if it's a "pass-through" part of the circuit. A junction box, where wires are connected or spliced, even without a device (receptacle or switch) has to be accessible. In that case, you really should run new wire in the wall to by-pass the box all together, from the feeder box to the next box in-line.
If this receptacle is the end of the circuit, only one hot, ground and neutral coming in, you still need to remove the box completely and find the other end of the wire and cap it off with orange or yellow wire nuts in the "pass-through" box that feeds it. Mark it as a dead end so the next person will have a clue where it goes. You can use a Sharpie and write right on the wire nuts or tape the wire nuts together with white tape and write on the tape.
If you have any doubt where the wire comes from, or where it goes or you just can't find the other end, get an electrician to come do the by-pass or the termination. It's well worth the expense, compared to the potential for a fire inside the wall, if this isn't done correctly.