Any views expressed within media held on this service are those of the contributors, should not be taken as approved or endorsed by the University, and do not necessarily reflect the views of the University in respect of any particular issue.

Magnus Hagdorn

Magnus Hagdorn

Research Software Engineer

Category: SysAdmin

The School storage is continuously increasing and datasets of many terrabytes are no longer exceptional. One of the big problems with large datasets is how to transfer them from one system to another. One of our research groups collects micro tomography datasets of the order of tens of terrabytes. These need to be transferred from […]

CephFS is now an integral part of the School of GeoScience’s IT infrastructure. It provides both personal and group storage for windows, mac and linux users. Since our first disaster due to running out of resources it has been running very reliably and it provided all the benefits we had hoped for. Until disaster stuck […]

Conda is the name of the package manager used with the data science distribution anaconda. It is available for Linux, Windows and Mac systems and is heavily used for managing scientific software environments. In the past we required our users to download and install conda themselves. Earlier versions of conda would modify the user’s environment […]

In one of my previous posts I outlined the ceph setup used in the School of GeoSciences. In this post, I would like to describe how the various bits fit together in some more detail. The diagram above shows a simplified version of our computing setup. Broadly speaking we distinguish between computers in the college […]

We had the opportunity to try out a cold start of all our servers located in the College Server Room on the weekend. We shut down all machines on Friday 11th June to allow contractors to work on the electricity supply on Saturday. From my perspective the most interesting aspect was shutting down the ceph […]

Now that our CephFS file store is in production and LCFG managed Ubuntu is available I began to experiment with running Samba on Ubuntu. The good news is that the LCFG samba component pretty much worked out of the box. The minor tweak that was required is going through the Informatics release process. Our test […]

Yesterday, I attended the ceph science meeting. I noticed the meeting was advertised on the ceph user mailing list and decided to go along to see what it is about. Ceph sysadmins from various institutions across the world attended to exchange information and discuss issues arising with scientific workloads. The meeting was very interesting. Our […]

The School of GeoSciences is running an aptly debian package repository. The following repositories are available: ubuntu ubuntu mirror inf informatics mirror uoe UoE package repository with component uoe world world package repository with component world geos School of GeoSciences repository with component geos All repositories and mirrors are available to the University of Edinburgh […]

This post describes in detail how we ended up with a damaged CephFs and our attempts to fix it. Background On 3rd September we scheduled some downtime to reconfigure the network of our ceph cluster. Prior to that we used two networks: one frontend network used by the ceph clients (and the rest of the […]

Providing secure, reliable, performant file storage is one of the fundamental services an IT team provides. This post describes in detail how the new CephFS based storage at the School of GeoSciences works. All our Linux machines, servers, compute boxes and desktops are running Scientific Linux and are managed using LCFG. We also use LCFG […]

css.php

Report this page

To report inappropriate content on this page, please use the form below. Upon receiving your report, we will be in touch as per the Take Down Policy of the service.

Please note that personal data collected through this form is used and stored for the purposes of processing this report and communication with you.

If you are unable to report a concern about content via this form please contact the Service Owner.

Please enter an email address you wish to be contacted on. Please describe the unacceptable content in sufficient detail to allow us to locate it, and why you consider it to be unacceptable.
By submitting this report, you accept that it is accurate and that fraudulent or nuisance complaints may result in action by the University.

  Cancel