It is your house, so if you want to lose the wall, then it is your choice. That settled, you have to ask "is is worth it for such a minor change?"
You will have to piece the wood floor. Can you match it?
You will have to fix the wall and ceiling drywall. Can you do it?
You will have to piece in the baseboard. Can you match it?
The outlet will have to be moved or removed. Do you know how?
Things to consider, yes?