This comes down to the raycaster. But I'm assuming you might be able to get away with reducing the distance of melee attack and add an additional line to check if the player is actually in view. When fixing issues dealing with AI, it means unfortunately getting down and dirty into the Lua of the AI modules. The AI modules have gotten pretty complex over time, becoming harder to modify.. and even more difficult since any changes affect all the of the AI for all enemies.
Probably the area to correct this issue is around here, in module_combatcore.lua:
module_combatcore.detectplayer(e,AIObjNo,PlayerDist,CanFire,detectstate,combattype)
if (PlayerDist < AIGetEntityViewRange(AIObjNo) and (g_Entity[e]['plrvisible'] == 1 or ai_bot_coverindex[e] == -2)) or g_Entity[e]['activated'] == 2 then
I dont like to mess about the AI much myself since it turned into modules. I still use some of the much older AI scripts which were unique per character type. Easier to modify. But they also come with legacy issues.
If you are changing any of the stock scripts for experimenting, make sure to make a backup first.
Store Assets -
Store Link
Free Assets -
Resource Link
i7 -2600 CPU @ 3.40GHz - Windows 7 - 8GB RAM - Nivida GeForce GTX 960