Our site has been hacked for the fourth time in two days. Find out how he
gets in, not just what he leaves behind.
A WordPress site was cleaned up. Twice by the malware scanner on the server, then once by hand, and twice more on a single day. Every time the attacker was back within hours. Fourth break-in in forty-eight hours.
That pattern is the clue in itself. An attacker who returns after a full cleanup does not need a new hole. He has left something behind that the cleanup does not touch. And the cleanup was about files.
The cause was in the database: a trigger on the comments table.
What a trigger is, and why this one is so nasty
A trigger is a piece of logic inside the database that fires automatically on a change to a table. On every new row, for instance. WordPress uses no triggers. Anything you find there was put there by somebody else.
This trigger was attached to the comments table. As soon as a comment or trackback came in with a certain sentence in it, the trigger created an administrator account. With a fixed password the attacker already knew.
That is where the trouble is. The attacker needs no file, no plugin, no access to the server. He only has to leave a comment on a public site. That is an ordinary visit, without a login, without anything that looks like an attack.
The logs let us follow the sequence. A trackback from one address. Fourteen seconds later a login as the new administrator, from a completely different address. Then, within a few minutes: all pages and posts into the bin, a file manager plugin installed, the theme and the plugins gone, the entire uploads folder gone, a fresh core plus a theme with malicious code in its functions file. The scanner emptied that file, and then the site returned an error instead of a page.
How you find it
One command per database. Ask for the triggers. If there is anything there, it is suspect, because there should be none.
In the same round, also look at scheduled events in the database and at stored procedures. Same idea, different place. This check belongs in every hack cleanup, and by now it is a standard step for us. The other steps are in what the hack check looks at.
There is a signature on the account itself too. The administrator such a trigger creates often has no e-mail address and a registration date from years ago. That last part is meant to avoid standing out in a user list, because whoever looks, looks at the newest accounts. An administrator without an e-mail address is always worth a question.
A trap while investigating: two time zones
Lining up comment dates and file times, we were two hours off at first.
WordPress stores the date of a comment in UTC. The times of files on the server are in local time. You then see a comment at one moment and a wave of file changes two hours later, and you conclude these are two events. It is one.
Convert everything to the same time zone before you build a timeline. Otherwise you go looking for a second break-in that is not there, or you miss the connection that is.
Restoring without bringing the trigger along
For the recovery there was one lucky circumstance: there was a backup the owner had made himself, seven minutes after the cleanup and before the wrecking. It still held a working site.
But that database dump contained the trigger as well. Of course it did, it was already weeks old. Restoring a dump and only then starting to look means you reinstall the backdoor, and the next comment hands over another administrator.
So removing the trigger belongs in the recovery script, right after the import, in the same step. Not as aftercare. That same dump also held spam posts, spam comments and a piece of PHP that was executed through a code snippet plugin. That went out in the same round.
Spam does not only sit in separate posts
Once the site was up and looked right, the home page itself turned out to be still polluted. Inside the content of that page sat hidden links, styled to push them off screen, plus a whole section of gambling text. No separate post, no plugin: it was in the content of a legitimate page, and it was already in the backup.
That taught us a step that is now always part of the job: after restoring a database, search the delivered HTML of the main pages for spam words. Not the database, the real page as a visitor gets it. That is how you also find what has been slipped into the content of an ordinary page.
Closing what the attack used
Cleaning up alone is no answer if the way in stays open. What belonged to this site:
Comments and pings off. The trigger worked on new comments and trackbacks. As long as a site does not need them, keep them closed.
Refuse trackbacks and the XML-RPC entry. That was the way in, and a site that uses no external clients loses nothing.
Invalidate all sessions and application passwords. A stolen session survives a password change.
Make the administrator password unusable and let the owner set it themselves. Then no password has to travel through a mailbox or a chat.
Block the addresses that were used. That stops nobody for good, but it takes the running attack off the site.
Common mistakes
Only cleaning up files. A backdoor can sit in the database. Ask for the triggers, the scheduled events and the procedures per database.
Trusting a scanner as the final verdict. A scanner looks at files. It did not see this trigger and would not see it.
Restoring a dump and then starting to look. Remove the trigger in the same step as the import, or you put the backdoor back.
Mixing times from the database and from files. Convert to one time zone, or you build the wrong timeline.
Looking for spam only in posts. Search the delivered pages, because hidden links often sit in the content of a real page. How spam survives a cleanup is in spam stays after the cleanup.
If you would rather not run these checks yourself, we keep an eye on your hosting. See web hosting and Invoker Assistant.