Ghost submitting my contact form ?

[studio9]

My contact form is being submitted every morning at the exact same time without any fields being filled in or the fields left as they are. This happens every single morning at the exact same time for last several days.

What can be causing this to happen? Could it be a search engine BOT visiting and triggering the form to submit ? Is it a bug with the hosting providers formmail ?

[djr33]

It could be an innocent bot if you have a link to the page. But in general they won't submit a blank form; they'd just follow links.

It is also possible that this is a not-so-friendly bot trying to spam you, but for some reason unable to submit any information in the form-- maybe your website isn't bot-friendly (which might be a good thing) so the submission is blank rather than filled with spam.

It's hard to know without more information.

What kind of contact form is this, or do you have a link?

If you're using PHP, I can help you to figure it out. When the email is sent with the contact information, you should add in, somewhere, the "user agent" string: $_SERVER['HTTP_USER_AGENT'];
That will give you some information, probably at least allowing you to identify/eliminate good bots (like search engines).

If it's a 'good' bot you can use "no-follow" on your link or you can set up a robots.txt file that asks them not to visit the submission page (though they could still visit the contact form to include it in search results).

If you don't get revealing information, then either:
1. It's a 'bad' bot, hiding its identify (that information is supplied by the user, so it's not guaranteed to be accurate).
2. It's not actually a visitor at all-- it might be a server error.

I can't imagine this being a server error, because I just don't see why that would happen. So only assume that if you can't figure anything else out.


Finally, once you do figure it out, you can probably eliminate these submissions by using a conditional ("if") statement based on the exact conditions-- the time of day? the user agent string? etc.



And if you're not using something you made yourself (as in PHP, etc.) then you might want to consider it so you have full control. If this is, for example, a "free form mailer", you may have no control over fixing (or even debugging) it. If so, it's up to you to decide if you want to replace it with your own code (which is obviously some effort) or keep receiving these false submissions-- it doesn't seem like a huge problem to me, if it's just once a day, but that could get annoying after a while.

[studio9]

the form doesn't require the fields to be filled in in order to be submitted.. I am not sure how to do that

I did try to put this in the submit button code yesterday ( rel="nofollow" ) but it still sent in another one this morning.. just not at the same time..

this is the form its using: http://www.hostgator.com/formmail.shtml

[djr33]

Your HTML is probably irrelevant. That won't change how the submit process works. All that would be on the server. (Exceptions include suggestions via Javascript, which might be ignored, and also the suggestion of "nofollow", etc.)

the form doesn't require the fields to be filled in in order to be submitted.. I am not sure how to do that

There are two ways:
1. Use Javascript to check if the form is filled in before submitting. This not secure and can be ignored. And it will have zero effect on bots (they don't use JS). There's a script here on DD for that if you want to add it. Usually it's helpful to remind users if they forgot something. But it won't actually prevent them (although it can make it a little more difficult).
2. Use a serverside language to check the required fields. You can't do that (at all) when you're using a free form mailer like that. You'd need to modify their code to do so. (Of course you can change your configuration so you are using server side scripting that you control, if you want. But it would be more work, and something new to learn.)

I did try to put this in the submit button code yesterday ( rel="nofollow" ) but it still sent in another one this morning.. just not at the same time..

It's hard to know if that could fix it.
1. If the bot already knows the URL, then you can't fix it by removing links to the page. It'll keep reindexing it when it wants to. That is, they may just be visiting http://www.mydomain.com/cgi-sys/formmail.pl
2. I don't see why the bot would be submitting your form empty. If it's spam, it usually includes dummy information and ads.
3. You could try to change the robots.txt on your site (google that) to ask the good bots to not go to that page. It wouldn't prevent the bad bots, but that could help you narrow down what's going on. I don't know how long that takes to starting working, though. I'm not sure if they check that file every time they load a page or not.
4. I'm not sure that rel="nofollow" works with forms, as opposed to links. In fact, good bots don't usually submit forms as far as I know. So maybe something else is going on here.



Is the form submission entirely empty? One way to test this would be to include a hidden field:
<input type="hidden" name="hidden" value="checkforbots">

Any real submission would include that value. (It's possible to change it, but real users wouldn't notice it-- it's only in the source code, and they'd also have no motivation to change it.)

If a bot is actually submitting your form, you would receive that value. If the bot is just visiting the submission page by URL (without submitting the form), then you would not receive that value. (Note that a false negative is technically possible if the bot removes that element from the form then submits it. A false positive is essentially impossible.) It's possible that the bot might submit the form but change the value of that field (sometimes spam bots guess what kinds of answers are expected to try to seem like humans), but you'd still get some value for it.

Do you get the fields, with empty/default values? That's also possible and means they are using the form and just filling in nothing.


Finally, you can add Javascript code that will do essentially the same thing. It can add a hidden field so that it will be submitted with the rest. This allows you to check whether Javascript is enabled for that user. If so, you'll get that field's value in the submission, and that means they're using Javascript and very likely human. If not, it's likely a bot.


At this point, it's probably a good idea to contact tech support from your host. They're the only ones that can solve it (such as blocking the IP address of the bot), and you can't do much from your site. At the very least they may be able to give you a little more information or let you know if this is a normal problem. I don't know how helpful they are (sometimes it can be hard to get a straight answers from hosting tech support) but it's worth a try.

[studio9]

the fields have values in them as default.. email field has "Your email here" here etc.. and when you click the field that vanishes atuomatically to allow you to enter your own text..

The form comes in with all of the default fields in place... so something is simply clicking the submit button without messing with the fields.

is there a way I can catch the IP address of the submitter with the form submission?

Another thing is that when it was submitted, I looked at my google analytics to see if someone visited the site but it didn't record any hits. Same goes for statcounter.......
Do BOTS get recorded by counters when they visit?

[djr33]

The form comes in with all of the default fields in place... so something is simply clicking the submit button without messing with the fields.

There are two possibilities:
1. A bot or person is visiting your site and clicking "submit".
2. A bot (or person) has saved a copy of your form and is submitting it to the same 'action' URL.
(You could test this easily enough by changing one of the fields, or, as above, adding a hidden field.)

is there a way I can catch the IP address of the submitter with the form submission?

Sure, but you'd have to do that on the server. You could try to do that by using a hidden field with serverside code to get the IP address. In PHP that would be:
[php]<input type="hidden" name="ip" value="<?php echo $_SERVER['REMOTE_ADDR']; ?>">[/php
The bot might choose to not send that, but it's somewhat likely it would work.
But then you'd need a PHP page (.php extension, PHP available on your server, etc.). A regular HTML page will work as a PHP page (just no extra features), so you wouldn't need to otherwise change your code.

The best way, though, again, would be to modify the receiving page, to be sure that you're getting it in the end, rather than routing it through the form.

And to stop it, you'd also need to use the receiving page.


Again, if you ask tech support, they might be able to help, even tell you the IP address if they keep logs.

Another thing is that when it was submitted, I looked at my google analytics to see if someone visited the site but it didn't record any hits. Same goes for statcounter.......
Do BOTS get recorded by counters when they visit?

Yes, but maybe no.
Bots don't always do the same thing on a webpage (eg, they will often just use the text version of a webpage, as code, rather than using a browser), but to your server they'll look exactly the same-- there's no way to tell them apart from how they view a page. It's the same requests, etc.
However, it's possible that your stats counter is designed to ignore requests from known bots-- using the User Agent string (optionally sent by the 'good' bots, and normal users) you can find out things like the browser being used or which bot (if any) if they choose to supply that info. It might say, for example, "google spider", etc. That would be the Google search engine spider going through your site (which is usually a good thing, but not if it goes to your submission page).



So your options are:
1. Accept that this may continue to happen. Do nothing.
2. Convert your submission to something that will allow you to modify the serverside submission process, such as using PHP. Then you can track the IPs and block them, among other things.
3. Contact tech support to see if they can do anything.
4. Try to use robots.txt to ask the bots to never go to your submission page http://www.mydomain.com/cgi-sys/formmail.pl (in fact, they probably shouldn't go anywhere in that directory).
5. See if that form mailer allows required fields. Not all do, but some do. You could require some field is submitted with a value. It's not hard to make that work, but most free form mailers don't have that kind of feature.

[studio9]

do you know a good simple free formmail file that I can just upload and would instantly work with my existing form?

[djr33]

You can google for some. In theory, what you have now is basically that. But you'd need to know how to edit it. Which language do you want to use?

You could relatively easily create one in PHP (or another language) that simply sends everything to you by email. But to have it do so in a 'smarter' way (organizing/verify/requiring certain fields) you'd need custom(ized) code anyway.

[studio9]

this is the one that keeps doing it i am pretty sure. Its last visit was the same time the form usually gets submitted..

Unknown robot (identified by 'spider') 222 hits.

I made the robots.txt file to ban visits to the cgi bin so I'll see if it blocks it in the morning.

[djr33]

Unknown robot (identified by 'spider') 222 hits.

What is the source of that information? If that's interpreted by your stats program, it might just not be known (but there would be more information in the original logs, at the very least an IP). But if that's all of the information it gives you, it sounds suspiciously like a 'bad' bot, or at least a not-so-important bot-- the ones you'd really want on your website will identify themselves clearly. Others, even for semi-legitimate websites may not be programmed well and honestly you may not care about some obscure search engine in China, India, etc. (There are a lot of non-English search engines that no one would ever use to find your site, in reality, and that might be what's happening here. If it's not just a spambot.)

I made the robots.txt file to ban visits to the cgi bin so I'll see if it blocks it in the morning.

That'll only work if it respects the rules. I'm guessing it will continue, but at least you'll know then.
If this bot is ignoring your robots.txt file, then you should contact tech support-- they can probably block it for you.