Press Shfit + Del. This keyboard combination will bypass the Recycle Bin and delete the file permanently.