At around 2:15am Sunday morning 9/25, the hosting company InMotion got hacked and Yakezie.com and FinancialSamurai.com went dark. Despite having just wrote the “dealing with burn out post“, I was burning myself out working from 11:30pm to 2:15am on things for the Yakezie Network and Financial Samurai!
Seeing the sites go down after spending almost 3 hours working reminded me of when my computer crashed the second I finished writing my 24 page final college English paper. What a disaster! I had just pulled an all-nighter, and all I could do was laugh. I decided I wasn’t going to let this ridiculous mishap ruin me so I proceeded to re-type all 24 pages again from memory. Four hours later, I was done. I don’t remember what grade I got. All I remember was that crap happens all the time, and it’s what we make of it.
When the crash hit this morning, a lot of things went on in my head:
* Am I going to lose all my work?
* I guess if the plane goes down (hosting company), even people in first class die (I’m paying for a dedicated server).
* Will my files be corrupted?
* Will they be going after my credit card info and passwords?
* Is there anybody else out there who can help?
* How much is a big bag of ganja nowadays?
* Are the bars still open?
* I want Belgian waffles with strawberries and whip cream.
* Ouch, why did I leave my tennis rackets on the floor to stub my toe with.
* So sleepy…. can’t stay awake.
After sending out a couple tweets and e-mails, I decided there was nothing really I could do, because I am a technical dumb ass. I sheepishly woke up Sydney from Untemplater at 2:30am to see if she could help, and she did by keeping me calm and deploying some back up protocols. I was able to pass out afterward. When I woke, I received an outpouring of e-mails and tweets from Challengers and Members a like, giving me a heads up and showing their concern and offering to help if they can.
Tom from Canandian Finance Blog and Glen from Free From Broke sent out some tips on where to find the Index.php file to replace and update. InMotion has confirmed that If you have some old Index.php file, just copy that to the existing Index.php, and your website will probably be back up. This is because hackers seem to have replaced Index.php files on many accounts. You can find all 9 index files for 3.2.1 here http://tomdrake.net/
Dana from Not Made of Money decided to go down the entire Yakezie Network list and e-mail all of us who were down because of InMotion. I asked Investor Junkie for advice on his hosting company, because he owns a hosting company!
Frugal Zeitgeist, Hope to Prosper, Sustainable Personal Finance, 20’s Finance, KNS Financial, Buck Inspire all reached out over e-mail as well.
Everybody was looking to help, and the person who was most instrumental in getting us back online was Suba from Wealth Informatics and Broke Professionals. Suba was the one who went into the trenches, did her technical magic and pulled us out of harms way! It wasn’t just a simple index.php replacement file for Yakezie.com, but an entire set of other things since the back-end of the site is so complicated. Thank you so much Suba! A total of 700,000 sites went down.
Here are some WordPress plug-ins to consider installing (were not installed before 9/25, but now are):
Exploit Scanner
Login Lockdown
WP Security Scan
Wordpress Firewall 2
Note: These plug-ins won’t prevent a back-end server attack, but at least they will do their best to prevent a front-end bulldoze attack. I like the WordPress Firewall 2 plug-in a lot. The “good thing” about this attack is that InMotion is probably implementing new protocols to prevent such a server attack again. It was also good to hear that so many people didn’t have to do anything, and that InMotion fixed everything themselves.
The Bright Side
I love the Yakezie Network because of the support we give each other. It’s really one of the most priceless feelings knowing that someone is out there, looking out for you. Whether someone has lost a job or is seeking some complicated advice, we got your back! We are doctors, lawyers, accountants, software engineers, designers, bankers, moms, and dads who make up the Yakezie Network. I literally don’t think there’s nothing we can’t resolve.
Supporting each other is what the Yakezie Network is all about. The relationships and friendships are what’s most important. Know that if you ever are in need of some help, just ask. Someone will be there to help you. Nobody will be left behind!
Thanks again everybody. May we all back up our files, establish contingency plans, and be vigilant. Getting attacked on-line can happen to any one of us. You guys make me so proud to be a part of the Network.
Best,
Sam
Glad to see it back up and running smoothly. Definitely a frustrating experience. Will this cause a change in hosting companies or do you think this will propel the company to get super secure and not have problems in the future?
That’s the irony. The safest place after a terrorist attack might very well be at the place of attack. I think Inmotion will get super secure, and hopefully give some reparation, otherwise, their business will suffer long term.
They should take the short term financial hit for long term goodwill.
How were the sites hacked? Where they ‘root’ed?
Donno mate, as I donno what you are referring too lol. But thanks for providing some answers wrt your company. I jus added you in the post.
Well if it’s because of the hosting provider’s inability that’s very concerning. If it’s because of your site and apps well then you need to make sure it’s kept up to date. Not only you should be using backups, but keeping sites with different passwords than other sites.
Also I recommend everyone on this forum to change their passwords and if you used the same for other sites (ie your bank) to change them ASAP.
Here is a thread on Web Hosting Talk Forum that discusses this hack.
http://www.webhostingtalk.com/showthread.php?t=1085368
Thanks for this. Can you give us a definition of the word “rooted”? And how does one do so? Assuming it’s much harder than the non rooted version? What are the solutions? thx
All modern operating systems have multiple levels of security. The administrators use a “super user” (or in ye olden days with Unix it was known as ‘root’) that has access to everything on the box. In order to make global changes like you saw on shard hosting and VPS (which I assume your sites were on), that means they were able to make these changes on a global scale.
The concerns are:
– more than one server was rooted. So that means they do dumb things like use the same root password through all of their servers
– their server software is not up to date globally and used the same vulnerability to get into all of them
– don’t have enough protective layers to prevent this escalating from to the level it did.
All of these are a bad sign and shows someone was asleep at the wheel within their company with security. Once rooted a hacker can do anything and all data on a server is suspect. Meaning they could leave trojans behind to get back in again after they fix the existing security hole.
So using the tools you mentioned are really for lower level hackings. ie you are running an outdated wordpress and the hackers broke in an defaced just your site.
This isn’t what happened so installing those tools, while help offer some protection does NOT protect you from a rooted server. The concern is:
– Did they really know how they got in
– Did they fix the problem
– and finally did the hackers modify anything else on the server?
Thanks for the info. Hopefully they’ll be working day and night to protect their clientele and scrub all the data! Otherwise, it’s bye bye InMotion!
I dealt with what looks to be the same hack on Evans site a month or so back and I actually found the back-doors the hackers created on his server..it was amazing what the little php file they left behind could do..when run from a web browser it generated a window with input boxes where you could input any command and it would be executed with root access on the server. It showed the root user and other info about the server..scary stuff. In Evans case this was the hosting provider using bad security practices that let the hackers in.
Yikes. What were the steps you guys implemented to prevent this from happening again? Is the hosting server more at fault, or WordPress? Hosting it seems.
Best network ever!
Thx for the concern and heads up buddy!
What a crazy 12 hours it’s been!! Glad I was able to calm you down and help you get some sleep last night. :) It’s always the worst feeling when our sites go down, especially when it’s hacking related. The good news is we’re back online and inMotion got a wakeup call to get their act together pronto.
You hit the nail on the head talking about how supportive everyone in the Yakezie Network is with each other. And Suba is a Yakezie Guardian rockstar!! Way to go Suba!!!
I’m glad you told me that inmotion’s site was down too this morning. I was thinking it might have only been targeted at my sites.
Appreciate the support! Hope you got some rest!
It was awesome how we got communications out between us to figure out what was going on. That’s what si great about a string network – you aren’t alone!
It’s funny because it was only about a month ago I told myself I needed to start backing up my entire site and I went out and bought an external hard drive. People Back Up Your Stuff!
Yeah, no joke about the backing up stuff. I have an auto daily backup, and at the time, I wasn’t sure whether my 3 hours worth of work would be lost or not. I haven’t checked on that post I was working on yet. Fingers crossed.
I will anxiously await to hear what inmotion’s press release is and how they will improve.
When these things happen, it’s always a good reminder for everyone to do a nice big backup to a secure location just in case. The small time commitment now could save a LOT of hassle later.
Very true indeed. Always a good reminder to learn from mishaps.
So glad to see the sites up and running again! Yeah Suba!!
It was awful to see that hacker message this morning. Definitely back up your sites!
This network is pretty amazing. Nice job getting things back up in such a short time. Thanks for sharing the behind the scenes clean up and a great reminder to backup files. Fascinating post. Kudos to you all!
Thanks… I was shocked today when I saw the outage. And disgusted with such activity. When my hard drive crashed a couple of weeks ago, and I lost lots, I went to my blog and immediately backed up the site. WordPress does have some helpful tools… but the ones you mention are new to me. Thanks.
Congrats to all the Yakezie network who work so well together and support one another. I’m proud to be a Yakezie challenger!
I’m glad to hear how everything got worked out! Sounds like you’ll sleep well tonight Sam!
I’m glad that everything worked out. The great thing about our network is that everyone is willing to pitch in, and there are so many areas of expertise!
Make sure you get some sleep, Sam!
I was glad when it came back on! Great work Suba! Nothing can keep Yakezie down for too long. Thanks for the tips about those plugins. I went straight to install them for my site.
Frustrating! I have been hacked before and it is amazing how awesome yakezie members are and how helpful they can be!
Glad everything worked out, and thanks for the very helpful information about WordPress plugins, too. I’m going to learn more about them RIGHT NOW and make sure we’re protected.
I had no idea! I changed my password for the site. Glad you were able to get things going again with the help of the network.
Nice team work. Suba has been a great tech help always, a day of blogging without Yakezie forum is painful. Sam have you thought of moving to Amazon cloud?
Aw, geez, I’m sorry that happened. I knew the site was down but had no idea why. Don’t you just hate technology sometimes?
Indeed! Love and hate technology both at the same time!
I can’t believe you are not using Mozy or Carbonite Online Backup. They are both very reasonable in their pricing structure and you will never have to worry about losing data again due to a hard drive failure or a hacked site.
I have all of my digital pictures and other documents secured through Carbonite and I sleep better at night knowing that if my laptop crashes or is stolen I won’t lose any of my files.
And No, I am not getting paid to recommend them.
I can’t believe it’s not butter… spray! Will check out those two services you recommended. thx.
Yep, I would definitely recommend using one of these services. I use Wuala myself but might like at these others since Wuala doesn’t cope too well with massive encrypted volumes (re uploads the whole thing every time
Glad to see everything is up and running even though I didn’t understand much of what you just said. At least if something bad happens, I know where to turn.
I am glad it worked out, it is definitely a wake up call for all us! I am glad the network is filled with resourceful people who can help in these situations.
What a fun day… I only had about 1 spare hour at my computer and got to spend it fixing 30 or so wordpress sites! Because of that, I never got a chance to submit to any carnivals this week, thanks hackers. Great work with all the emails sent around and a couple skype messages too. Awesome group of bloggers!
Tom, you have 30 or so wordpress sites?! Or, are you working with that many other sites, as that is a lot! I think I’d go crazy :)
This was a massive hacking job because I have friends who had some of their websites go down. First thing in my mind would be ‘did I just lose all my work?’ I’m glad everything is up and running for you and your sites.. would have been a stressful time! Ive had only one hacking incident.. pain in the butt!
Belgian waffles with strawberries and whip cream sounds amazing!
I saw everything down this morning that the first thing I did was email Suba! Glad everything is back up!
I could not believe what I was seeing yesterday when that window came spiraling into my screen and said you’d been hacked! I’m glad to see that everything is fine now, and your site is back to normal. Scary!!
Oh no! The sad fact of the matter is, if someone wants to hack your blog bad enough, they are likely to find a way in. Pleased you were able to get it sorted so quickly though – what a relief!
Incidentally, I’m attending a half-day WordPress security course later this week, for work. Perhaps I should report back?
Yes, definitely report back and share what you’ve learned! Any things learned on prevention and security bolstering tips for us bloggers would be great!
Glad to hear everything came back up. I do weekly backups of my blog though I really have no idea how it might work if push came to shove, since things (so far) have been relatively calm.
Wow–scary, and yet there was such an outpouring of help! Thank you to everyone who fixed the problems. I tried to get onto Yakezie.com yesterday morning three times and could not get through; I mistakenly thought Suba was working on the site because not many people get onto websites on Sunday mornings.
I hope everyone (especially Sam) is catching up on sleep! Thank you to Suba, and all the other tech-wizards out there.
No sleep for the weary! It was back to 1:30am sleep this Sunday and 6:30am wake up again this morning. I think I’ll take an afternoon siesta, Spanish style!
Sorry to hear about the problem, we were out all day Sunday. I hate dealing with those hackers too. Great job pulling together team!
Dealing with hackers is tough. I kind of get the hacker culture enough to understand why they do this, but I too wish they would put their skills to better use and go to a hackathon to create something amazing from an API or create a new app.
I read Tiger Mates (the hacker)’s interview on thehackernews.com and he seems like a normal guy who is just bringing about awareness and not trying to steal or permanently take down anything. It is still painful for the 700,000 sites that went down.
Umm, he could have just emailed or called the company rather than do what he did. Or hacked on a much more benign level (like maybe just the company’s home page).
Indeed. So, a lot of it is an ego thing I have to imagine. He who can take down the most sites at once.
There are many benefits to having a good network with people of varied talents. This just proves it. And this is a good reminder to each of us to back up often.
Thanks to everyone who helped Sam get the site back up and running. You are all amazing.
I can’t believe we got hacked to begin with. Scary stuff!
Like others have said- this network rocks!
I’ve been dealing with an attack as well, a network of pcs across the world are posting spam comments using urls such as google.com, bing.com. yahoo.com. What is the point of promoting search engines? I don’t know how they were able to decipher the capcha, I changed it to a math equation which helped calm things down.
I’m getting the same spam. I have no idea what the point is. Spammers seem to be going on OVERDRIVE lately!
Glad everything is back up and running Sam! I was nervous to see how bad it was, and I definitely backed my things up right after. Go Yakezie! Always looking out for each other.
Wow, I had no idea… this is crazy. I just heard about the hack from Investorz’ Blog and headed over to see what’s up! It looks like things are already taken care of; glad to see you had so many people around to help out, Sam!
I’m glad you and the network were able to resolve this so quickly. I’ll be installing the WP security plugins you recommend. That black screen of hacked death gives me a terrible sinking feeling, and I want to avoid it if possible.
Hi Sam,
This is Brad with InMotion Hosting. I just want to apologize to everyone here who’s site was hacked. Out own site was hacked as well, so we definately understand how you feel.
I’m hoping to include a link here for any of our other customers who may be reading your post. If their site is still defaced and you need help with fixing it, please see:
http://forum.inmotionhosting.com/viewtopic.php?f=57&t=37822
If curious, you can also read a message from our company president on this issue:
http://forum.inmotionhosting.com/viewtopic.php?f=57&t=37821
Thanks,
– Brad
Hi Brad,
Thanks for reaching out. More importantly, it’s what your firm decides to do with the affected clients that matter. We’re all in this together, however we are your clients who rely on you to keep us safe.
If we cannot rely on you at your server level, then we will all disappear. I really suggest doing something great, taking a short-term financial hit, to build long-term relationships. Your competitors are chomping at the bits, offering incredible incentives now.
Hold on to us.
Regards,
Sam
Hi Sam,
Thanks for getting back with me.
We’ve always put our focus on Customer Service, and we are fully aware of all the headache and frustration that our users are going through. Please do let me know if you receive anything but exception customer service when working with InMotion Hosting to resolve the issues at hand.
Thanks again,
– Brad
Brad, my associate has shot you an e-mail. Hope you respond and offer something up. Thx, Sam
I will NEVER EVER EVER use Inmotion service again!!! Never!
I was looking for site security and performance when I moved from GoDaddy shared hosting to Inmotion VPS about 10 days ago. It took me some time to transfer and setup 12 sites. And what I got? All 12 sites defaced overnight!
Well, I was lucky enough to have all backups with me and got my sites up and running but it was so frustrating.
For two days I was watching my sites very closely and everyhting looked good.
This morning I woke up and logged in into my statcounter and found… I have no clicks at all!!! Panic!!! Couldn’t login to cpanel, whm, ftp. Panic!!!
A minute later I found this email from inmotion:
I tried calling them but they didn’t pick up the phone because they are busy fixing their crappy servers.
These stining rats disabled my account!!! All 12 sites!!! Without any warning!!!
I never had any performance issues while hosting with GoDaddy.
Some of my sites are getting over 10K clicks/day and I didn’t have any major issue with basic shared hosting account on GoDaddy.
I’m back to GoDaddy now but my hands still shaking.
Inmotion hosting sucks big time.Failed!!!!
Sam, I can totally relate. Same thing happened with GoDaddy a few weeks back and I had 2 days + time + money getting going again (technical dumbass, too). My favorite part of this post was “what’s going through my mind.” Sam and all fellow Yakezies, thank you for keeping this awesome network going.
Non-technical savvy users are easily fooled by web hosting companies who falsely explain how far a total compromise could go. Security-minded guys will give you the following conclusions.
1 – InMotion said the goal of this mass hack is just to do defacement.
These hosting guys never know hackers have installed rootkits and backdoors for future access.
They think that it’s safe and simple as restoring clients’ web sites from backups.
Once a box is hacked at the root level, it can’t be trusted any more.
2 – Hackers could have compromised the inMotion several weeks/months before. Finally, they’ve been aware that the exploit they use have been discovered/known by other same-minded hackers. They do mass defacement to notify inMotion guys to patch this hole.
We’ve seen mass hacking these days are not just for fun and fame. They have been used for generating revenue in black markets. Now, some clients are ready to move to other hostings. Others are just staying at inMotion and hoping for this mass hack not to happen again. Rest assured, this hack will not come back as hackers may now have future access at their will using backdoors that ultilize steathy covert channels to remotely do malicious stuffs.
Stay Secure.
[…] functionality, nearly all updates contain some security fixes. We have all seen the results of our hosts being hacked. We have discussed security plugins in the forums. You can read all about WordPress security at the […]