Data Management
- Scott Gallagher
Overview
The Data Management page serves as a comprehensive guide for efficient data handling and secure backup strategies within the College of the Environment. This guide is designed to assist staff, faculty, and students in managing their data in a manner that aligns with institutional policies, enhances collaboration, and ensures compliance with regulatory requirements.
Key elements of this page include an overview of different data storage and backup solutions such as Local Storage, External Drives, SharePoint, and Ceph Cluster. Additionally, it outlines data classification, backup frequency, and retention policies to ensure optimal security and disaster recovery.
Purpose
The purpose of this page is to offer practical guidance on managing data across the College, helping all members maintain data integrity, security, and compliance with institutional policies. The guide aims to:
Provide insights into various backup solutions and their appropriate use.
Clarify data classification to determine the correct backup and storage strategies.
Offer best practices for collaboration, data sharing, and version control.
Outline data retention policies and backup schedules.
This page functions as a centralized hub for all data management inquiries, offering tools, resources, and FAQs.
Backup Methods
The College uses multiple storage and backup solutions, each suited for specific data needs. Below is a breakdown of the key methods and their recommended use cases:
Local Storage
Usage: For sensitive or offline data. Files stored directly on a user's device.
Backup: Manual backups are recommended, using external drives or cloud solutions.
Retention: Keep only what is necessary; archive periodically.
External Drives
Usage: Best for temporary offline storage and backups (e.g., Time Machine backups).
Backup: Regular manual backups required. This is not a primary solution for large-scale data storage due to security risks.
Retention: Short-term storage only. Not ideal for sensitive or long-term data storage.
SharePoint
Usage: For structured team collaboration, project files, and department-level documents that need to stay within the College. Files here can be deleted upon faculty departure when specified.
Backup: SharePoint includes built-in version control and automated backups.
Retention: Data is subject to institutional retention policies. Files are automatically archived and deleted unless otherwise specified upon user departure.
SharePoint Sites for Faculty
Urban & Environmental Planning & Policy: https://wwu2.sharepoint.com/sites/UEPPFacStf/SitePages/CollabHome.aspx
Environmental Science Faculty & Staff: https://wwu2.sharepoint.com/sites/ESCIFacSf/SitePages/CollabHome.aspx
Environmental Studies Faculty: https://wwu2.sharepoint.com/sites/ENVSF/Pat%20Buckley/Forms/AllItems.aspx
You can access these from the main SharePoint page for the College: https://wwu2.sharepoint.com/sites/CENV under the Faculty & Staff Storage dropdown.
Ceph Cluster
For large-scale data storage, particularly for research data and long-term archives, Ceph is the primary solution. Access to Ceph shares is controlled through Active Directory (AD) security groups, and share management involves creating and assigning appropriate permissions.
Share Management: After creation, shares are managed by specific groups responsible for permissions. For example, administrators like IT Managers or Help Desk Agents are designated, with group memberships assigned for managing access.
Permissions Setup: To manage permissions effectively, AD security groups are assigned, such as
sec.its.eis.ceph.[SHARENAME].mgmt, which allows EIS to manage access.Technical Setup for SMB Permissions: More detailed instructions for SMB permissions management can be found on the following Confluence pages:
Data Classification
Classifying your data helps in determining the appropriate storage method and security level. Use the following guidelines for classification:
Confidential Data: Use local storage or Ceph with restricted access.
Internal/Institutional Data: Best suited for SharePoint.
Public Data: Can be stored on SharePoint or Teams.
Data Backup, Retention, and Cleanup Policies
To ensure data security and prevent data loss, it is essential to establish regular backup routines and implement retention policies across all storage solutions. This section outlines best practices for backing up data and maintaining data integrity through periodic cleanups.
Backup Frequency and Retention Policies
Local Storage: Backup critical files weekly to avoid loss of important data. Older files should be archived quarterly to free up space and maintain efficiency.
SharePoint: Automatic backups are managed through built-in versioning, ensuring compliance with retention policies set by the institution. SharePoint also provides robust options for file recovery and history tracking.
Ceph Cluster: Large-scale data, especially research and long-term projects, should be backed up monthly. Annual archiving is recommended based on project timelines and departmental needs to ensure that essential data remains accessible for the long term.
Retention Policies
Retention policies for data stored in Local Storage, SharePoint, and Ceph are still under discussion and will be finalized in collaboration with EIS, Mariah, and the Faculty. These policies will outline how long data will be kept and when it will be archived or deleted, ensuring compliance with university and grant regulations.
Collaboration and Sharing
Internal Collaboration
For internal teams, SharePoint and Teams are the primary tools for collaboration. Best practices include:
Set clear access permissions.
Use version control to track document changes.
Regularly review shared files to maintain accuracy.
External Collaboration
For external collaborators, only grant temporary access when needed, using SharePoint’s secure sharing features.
Tools and Resources
Available Tools
OneDrive Sync: For personal file access and offline capabilities.
SharePoint: For collaborative and department-level data storage.
Ceph Cluster: For large-scale, distributed data storage.
SMB Share Access: For managing access to SMB shares. Access is controlled by AD security groups, and requests must be approved by designated data owners or technical contacts.
Training and Support
FAQ
Why are we moving from local servers to Ceph and SharePoint? We aim to enhance data security, streamline collaboration, and reduce reliance on aging local servers.
Why did we remove the servers? The servers were aging and required constant maintenance, which delayed critical tasks. By transitioning to cloud-based solutions like Ceph and SharePoint, we reduce the risks associated with server outages, improve data accessibility, and enhance security with automatic backups.
Why can't we log into OneDrive on shared computers? OneDrive is intended for individual use with personalized login credentials, ensuring data privacy and security. Shared computers pose a risk for unauthorized access to your personal files. Additionally, OneDrive downloads data to the local machine, which can create storage issues on low-capacity drives. For these reasons, OneDrive is not configured for multi-user environments.
Where is my data that was on the servers? Your data from the servers has been securely migrated to either OneDrive, SharePoint, or the Ceph Cluster based on its classification and usage needs. Please check with your department or IT if you're unsure where specific files were moved.
What happens to my data on local servers? Your data will be assessed, cleaned up, and migrated to OneDrive and SharePoint. Data deemed necessary for physical archiving will be securely stored.
How do I organize my files before migration? Create a clear folder structure in OneDrive that reflects how you want your files to be organized in SharePoint. Categorize your files into active projects, archives, and personal data.
How do I request access to the Ceph Share? Please contact [Admin Name] to request access, specifying your project needs.
Will I still be able to access my files offline? Yes, with the OneDrive sync client, you can access and work on your files offline. Changes will sync automatically when you reconnect to the internet.
How can I ensure my files are secure? Review and adjust file permissions, especially for sensitive data. SharePoint provides robust security features to protect your data.
What are the benefits of using SharePoint? SharePoint offers enhanced collaboration features, centralized document management, and easy sharing across departments, streamlining workflow and communication.
What happens to my data after I leave the College? Any data stored in SharePoint can be archived or deleted follow the retention policy.
What if I need help during the migration? I am available to provide one-on-one assistance throughout the migration process. Please reach out to me for any support or questions you may have.
How often should I clean up my files after migration? Regular audits and cleanups are recommended to maintain an organized and efficient data environment. Implementing data lifecycle policies will help manage retention and deletion.
Can I use external hard drives for backup? While external drives can be used for temporary backups, they are not recommended.