UKC02 – Service Outage 08/07/2017

UKC02 – Service Outage Updates


Latest Update

Sun 16th July 1024 : The server data restore is complete, we have provided a list of the commonly reported issues below:

Plesk Default Page

Upon completion of the restore there were a number of websites displaying the Plesk Default page, this has been resolved. If you have a ticket open regarding this issue, please try accessing your website again and let us know if you able to / unable to access your website.

Email

We have applied the new server encryption key to all mail account passwords, you will now be able to login to webmail / Outlook using the most recent password set on your mail account.

Plesk Login

We have resolved the following error which was previously displaying inside the Client Portal, you will now be able to login to Plesk from inside the Client Portal:

Error code: 0. Error message: Unable to find appropriate manager for this version of Panel. Plesk should be at least 8.0 version.

MySQL Databases

If your database restore is incomplete, please submit a ticket to the HelpDesk and we will schedule a full restore of the affected database.

We have scheduled an upgrade of MySQL from 5.1 to 5.5.49 to be completed overnight to match the version that was previously installed on the server.

If you are seeing the following error in PHPMyAdmin please login to Plesk and reset the database user password for the database that you wish to access PHPMyAdmin for:

Connection for controluser as defined in your configuration failed.

500 error / Blank page

Upon completion of the restore the PHP version was set to 5.3 on all domains, we will be changing the PHP version to 5.4 for all domains on UK02 today to match the version that was previously used. This will resolve most reports of the 500 error / blank page. If you were previously using PHP 5.6 please let the HelpDesk know and we will apply PHP 5.6 to your domain.

If you have logged a ticket and have not yet received a response, please rest assured we are aware of the issues that you have reported and are currently working them.

1: PSA database – COMPLETE
2: Restore vhosts – COMPLETE
3. Restore emails – COMPLETE
4: Restore databases – COMPLETE
5: Restore DNS zone files – COMPLETE

Thank you very much for your patience throughout this difficult time.

Will I be compensated for the downtime? Yes! Even those who have not purchased a Backup Plan or Business / Unlimited hosting service will receive some form of compensation in the form of discount codes, discounted renewals and free hosting extensions. Full details will follow once we get everything working.

Who is assigned to this incident? Our best engineers together with the most senior technical resources of our upstream DC/Server partner EUKHOST are working to bring this matter to the fastest possible conclusion.

Why is it taking so long to solve? The sheer amount of data we are working with takes considerable time to restore and verify data integrity. Each restore cycle takes up to 20 hours and unfortunately the failure experienced has required multiple restoration cycles.

What are the priorities? Our primary objective is to retain 100% data integrity for our clients. The fastest method to obtain this goal together with bringing the server back online safely and as quickly as we can are our objectives.

I have my own backup, can I get an account on a new server? Yes, please let our helpdesk know and you can have a new account setup on a new server and perform your own data restore using the Plesk backup/restore tool.

Wed 12th July – 0708 : In unison with our upstream DC/Server partner EUKHOST, another attempt is in progress to restore all data to a newly built server. Data recovery is showing 31% complete. This will be our final attempt before we switch to a manual site-by-site restore process. We will post an update at 1800. We are overwhelmed by your support and appreciate your patience during this difficult time.

Tue 11th July – 1150 : A full data restore is currently nearing completion. This will be followed by a thorough data integrity check. If all checks pass then the server will be brought back online later today. We will post an update at 1800. Thank you for your continuing patience.

Tue 11th July – 0604 : A full data restore is currently nearing completion. This will be followed by a thorough data integrity check. If all checks pass then the server will be brought back online later today. We will post an update at 1200. Thank you for your continuing patience.

Mon 10th July – 2249 : A full data restore is currently in progress. Due to the large amount of data the process is expected to take 8 hours from this point. A data integrity check will then follow. We will post a progress update at 0800. Thank you for your patience.

Mon 10th July – 1810 : A full data restore is currently in progress. Due to the large amount of data the process is expected to take 14 hours from this point. While the restore is progressing we are investigating options that could speed up the process. We will post a progress update at 2300. Thank you for your patience.

Mon 10th July – 1134 : A full data restore is currently in progress. Due to the large amount of data the process is expected to take 20 hours from this point. While the restore is progressing we are investigating options that could speed up the process. We will post an update at 1800. Thank you for your patience.

Mon 10th July – 0600 : A full data restore is currently in progress. Due to the large amount of data we expect the process to continue throughout the day. We will post an update at 1200. Thank you for your patience.

Sun 9th July – 2300 : A full data restore is currently in progress and will run throughout the night. Upon completion, comprehensive and exhaustive data integrity checks will be performed. We will post a progress update at 0600. Thank you for your continuing patience.

Sun 9th July – 1600 : We are preparing for a full data restore from the data set taken 0400 on Saturday 8th July. The process will begin this evening. We will post a progress update at 2300. Thank you for your patience.

Sun 9th July – 1100 : The Server Recovery Team have requested more time which we have granted. We are now prepared to run a full data restore from the most recent data set if required. Once the restore process begins we will have an ETA for its completion.  We will post an update at 1600. Thank you for your patience.

Sat 8th July – 2300 : The Server Recovery Team continue to work with the volumes and configuration checks which are going to run over night and in to tomorrow morning. If they not satisfied with the results we will begin a full data restore from the most recent data set taken 0400 on Saturday 8th July. We will post an update at 1100 tomorrow morning. Thank you once again for your patience.

Sat 8th July – 1800 : The Server Recovery Team continue to work with the volumes and configuration checks which could lead to a speedy recovery. However, it is possible we may need to perform a full Data Recovery from our Backup System. The most recent data set we have is 0400 Saturday 8th July. We still hope to avoid a full data restore (it’s a lengthy process so we we hope we can avoid it) however in the meantime it’s important for us to verify exactly what went wrong, to ensure we do not inherit the problem going forward. We will post an update at 2300. Thank you for your patience.

Sat 8th July – 1400 : The Server Recovery Team are currently performing an exhaustive check of the server drive units and controller. This process will take a number of hours. We will post an update at 18:00. Thank you for your patience.

Sat 8th July – 1000 : Server UKC02 has failed to start up successfully this morning after a restart. The issue is now being investigated by the Server Recovery Team. An update will be posted at 1400 hours. Thank you for your patience.

Customer Service Outage and Security Audit

Billing Portal Outage and Security Audit

Late on 10th January we were alerted of a potential security breach of the web server which hosts our primary site and helpdesk/billing portal. We are currently assessing the situation and undergoing a security audit to contain and resolve the situation.

As a precaution, we have temporarily disabled our Client Panel whilst we perform audits and upgrades. Note that we do not store financial information such as credit card details on our systems.

The situation is still developing. As an additional precaution, we have reset all client Plesk panel passwords. To gain access to your Plesk panel, please perform a forgotten password request. Weak passwords will no longer be allowed.


Ongoing Updates – Please Check Here for Updates

Check here for updates. Use our Live Support facility to speak to our technical staff for urgent support issues.

We are launching our New Billing and Helpdesk Portal today

16/01 13:00 – Our new Billing and Helpdesk Portal is now online! Please test your login, have a look around. Let us know about any outstanding issues. Or contact Live Support if you prefer as always. Thank you so much for your patience during this service outage. Please continue to be patient with our support staff who are working very hard to help with your problems.

16/01 06:00 – We are pleased to announce that the new Billing Portal installation, upgrade and migration is complete! Before we go live we are awaiting the full propagation of a DNS change. In the meantime we are testing every feature. We do not expect any further issues and our objective is to activate the new portal at approximately midday today. Our helpdesk will be back in just a few hours. Please continue using our Live Support for urgent issues. Thank you for your patience during this upgrade.

14/01 – We have restored nameservers on all affected domains, you should start to see your website and email services come back online within a few hours, please note that it may take up to 24 hours for the nameserver changes to fully propagate.

13/01 -Friday/Saturday – We continue to apply custom name servers to the impacted domains which were left unlocked by clients. We will continue this process through Saturday. Once again, please contact Live Support for urgent issues only.

12/01 16:00 – The new billing server hardware is in place and migrations will start overnight. We have replaced everything. We are projecting to have the client billing service back online within 24 hours. Thank you for your patience.

11/01 23:00 – Web Hosting and Email – We are currently working through all accounts and are bringing them online as we can. Most accounts are not affected. We suspect it may take a number of days to fully complete our audit of every account.

11/01 20:00 – Name servers have been applied to each unlocked domain according to the server it was on. If you were using custom nameservers (eg. your domain was not parked) please send the correct nameserver settings to nameservers@uk-cheapest.co.uk.

11/01 12:00 – Unlocked Domains – A number of domains that were not locked had their nameservers changed without permission. If your nameservers are not correct, please email the correct settings to nameservers@uk-cheapest.co.uk. Please include the domain names and the nameservers that you would like applied to each domain in your email. Please ensure that the email is sent from your account email address.


Your website and data are not affected. If your site is offline, please advise the Live Support advisor of your domain name and we will investigate further. Please note at this time, basic support will be limited. Only contact us if your site is down or urgent.

As a precaution, all WHMCS and Plesk Client passwords have been reset. Please do a forgotten password request to set your own password before contacting Live Support.

Once again, thank you for your patience during this difficult time.

Kind Regards,

UKC Support Team

Which characters can I use in a domain name address?

Regardless of the extension, all domain names use the same character rules which must be followed when you search for yours:

You can use letters (a-z), numbers (0-9) and dashes/hyphens (-). Spaces are not allowed, and your domains cannot begin or end with a hyphen. Your web address cannot include special characters like “&” or “!”. However, there are exceptions with IDNs where you are allowed to include special characters from other languages in your domain names.

A domain name cannot be less than three characters long, not including the extension, and names cannot be more than 63 characters so check yours is less than this.


UK Cheapest Domain Names



Where can I register .LONDON domains?

You can register .LONDON domains at UK Cheapest!

We offer a huge selection of domain name extensions, including .LONDON and many others.

You can register your domains from 1 year to 10 years. We accept all forms of payment for domain registration including PayPal, all major debit and credit card suppliers and Internet/Online Banking payments.


Buy .LONDON Domains



What details do I need to register a domain name?

To register a domain name you will need to provide a few details such as your full name, your postal address and some other contact details. It is important that the information you provide is correct and up-to-date in case of a dispute with your domain name or a situation where you need to show proof of ownership. These details are also important when it comes to renewal of your domain names.


Register Cheap Domain Names £4.99