Most of our food exports consist of highly processed products rather than raw commidities. That suggests that farmers sell their products elsewhere.
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/515048/food-farming-stats-release-07apr16.pdf
You might be right, but I don't see how your second statement necessarily follows from the first one.
Aside from domestic consumption and exports of raw commodities, presumably some farmers sell raw ingredients (off the top of my head: the grains needed for Scotch whisky, for example) to UK manufacturers who then export the processed products.
I think it would be misleading to assume that those farmers (the raw ingredient suppliers to UK manufacturers) wouldn't be affected by a no-deal. IMO, they would be as tariffs for non-EU members on processed foods are generally considerably higher than for "raw" (fresh / frozen) produce.
Norway (non-EU, but with EFTA benefits) exports fresh salmon to Poland for processing. Lower wages are one factor but so is the export tariff which is lower (2% for fresh / frozen) versus 13% if it were to export the salmon already smoked. As an EU member, Poland can then export the smoked salmon to other EU countries tariff-free.
https://uk.reuters.com/article/uk-britain-eu-norway-salmon/norway-satisfies-eu-smoked-salmon-appetite-through-polish-back-door-idUKKCN0WI1VHThere is also the issue of tariff quotas (higher tariffs for nationwide exports of whatever over a certain quota), plus seasonal tariffs on fresh produce. I'll add a link if I find it again, but for example the tariffs for lemons / oranges from non-EU countries are much higher when EU produce (e.g. from Spain) is ripe for picking (winter for lemons). Off-season (summer in the northern hemisphere) tariffs for non-EU countries are lower.