No. You don’t need to pick inedible fruit.
Out of consideration, please let the homeowner know that you aren’t able to pick their fruit and why.
If it is under-ripe suggest a future date when you can go back to pick the fruit.
Need help determining if it is ready? Click here to learn more about judging ripeness of different fruits.