Saturday, 28 June 2014

SRDF Asynchronous (SRDF/A)

SRDF/A is a long distance disaster restart solution from EMC with fast application response time. In SRDF/S as the host gets the I/O complete message from the primary array only when the I/O is written successfully at primary as well as Secondary array cache and an acknowledgement for the same has been provided to the primary array and the host as well. Now suppose if these 2 array are located very far from each other geographically, it will certainly have the impact on the performance for the host and application as well. So for long distance solution SRDF/S is not a desirable solution. To overcome this and to provide a faster response time, EMC has come up with another long distance replication solution called SRDF/A or Asynchronous SRDF.

So what happens exactly in SRDF/A ? In SRDF/A, primary array does not wait for the acknowledgement from the secondary array, rather it sends the acknowledgement back to the host as soon as it completes writing data to it's cache and later on data transmission will happen over the SRDF links. In this operation, host write I/Os will be accumulated in the delta sets in the primary and secondary array cache, dependent-write consistency of each delta sets will be managed and delta sets will be transferred across SRDF links in cycles.

So let's talk about delta sets and cycles. These delta sets and cycles are very important to achieve dependent write consistency and is possible only by the processing of the ordered delta sets and cycles between the primary array and secondary array.

Before we discuss further, have a look at the below diagram.




Now let's take a look at how the actual operation takes place. Enginuity has actually divided delta sets primarily in 4 types, "Capture", "Transmit", "Receive" and "Apply" and divided cycles in to 2 types, "Active" and "Inactive".

  • The "Active cycle" on the primary array will have the current host writes or "N" data updates in the "Capture delta set".
P.S. "N" means current data, "N-1" means data in the previous cycle and "N-2" means data before the previous cycle.
  • The inactive cycle in primary array will have "N-1" data set that is transferred to the secondary array.
  • The primary "inactive cycle" is the "transmit delta set" and secondary array "inactive cycle" is the receive delta set and will be having the "N-1" data sets.
  •  The active cycle on the secondary array contains "N-2" data updates in the "apply delta set". This is guaranteed dependent-write consistent image in the event of a disaster or failure.
There are various features used by SRDF/A to avoid the link loss, to improve performance and to recover from a failure scenarios. For example:

  • Reserve Capacity
    • Transmit Idle
    • DSE Pool
  • Group Level Pacing
  • Device Level Pacing
  • Write Folding
  • Write Pacing
  • Tolerance Mode
I will be talking about these in coming posts.

Wednesday, 25 June 2014

SRDF Synchronous or SRDF/S Operations

As I have already described SRDF in my previous blog, there are various modes of SRDF operations and here in this blog, I will be discussing about SRDF synchronous or SRDF/S mode. But before we understand this, it would be wise to understand the term "Enginuity Emulations" as I will be using this term quite often in this blog and in my posts on SRDF.

The Symmetrix Operating System which is famously called as Enginuity Operating system, provides executables that manages I/O execution, data protection, data availability and integrity. These executables are called Enginuity emulations. Every processor in each and every director of the symmetrix array will have specific emulation codes to handle various operations. And this makes an independent director to act as a cached disk array in itself.

Let me give you a brief on the these important Enginuity Emulation which will help in SRDF I/O flow.

  • Host Emulation: Manages I/O between host and the symmetrix array, moving data between the host interface card and the cache in the primary array.
  • Drive Emulation: Manages the data movement between the cache in the array and the disk drives in a single array.
  • SRDF Emulation: Manages the data movement across the SRDF links between the cache in the primary array to the cache in the secondary or remote array.

Below diagram will give you a picture of the various emulations, we have discussed above.




Now as you know the emulation, let's discuss the SRDF Synchronous mode. In SRDF/S operation, R1 devices are mirrored to the R2 devices in real time. Real time mirroring means that host updates are propagated to R2 devices in a regular continuous I/O flow. This mode will be defined at the device level and  both the primary and the remote devices should be operating in this mode for it to work.

Write Operation in SRDF/S mode: 

Whenever the host issues a write operation to the primary array, this operation will only be completed once the the write update has been propagated to the cache of secondary array and an acknowledgement of the same was received by the host. Entire operation takes place in below steps :

  1. Host issues a write I/O to the primary array A.
  2. Host emulation will make sure that data is written to the cache of Symmetrix A and also creates a write request:
    • For the drive emulation to de stage the data from the cache to the R1 local mirrors.
    • For the SRDF emulation to send the I/O over the SRDF link to cache in Symmetrix B.
  3.  Now this write request created by host emulations in step 2, will be received by the SRDF emulations and it in turn access the updated data from the cache and frame it according to the SRDF protocol and send it across the SRDF links to symmetrix B.
  4. SRDF emulation in symmetrix B receives the data from the SRDF links, write it to the cache and sends an acknowledgement back to SRDF emulations of symmetrix A. It also creates a write request for the drive emulation in symmetrix B to de stage the data from the cache to the R2 local mirror drives.
  5. SRDF emulation forward the acknowledgement back to the host emulation, which completes a the successful I/O flow by sending a a successful command completion status to the host. 
Remote Mirroring: 

After data reaches in the cache of secondary array B, drive emulation will create a request to de-stage the data from cache to the R2 local mirrors. SRDF remote mirroring provides additional protection from the failure or disasters in case of multiple drive failures at the primary site. If all R1 local mirrors fails during host write operation, the wrote I/O proceeds to the secondary array via means of SRDF links. Now as R1 local mirrors are not receiving the data due to failure and entire data is being stored at the R2 local mirrors, this data at R2 local mirrors will be marked as invalid tracks indicating that the data at R1 side is not current and should not be used by the host.

If no new host write I/O to the R1 device have arrived since the last failure, R2 device will automatically synchronize it from itself as soon as the R1 device comes on-line as R2 device contains the invalid tracks.

Read Operations: 

In normal read I/O operations, SRDF emulation is not required. As soon as the read I/O request is there, host emulation will check the data in the cache and returns it back to host if it exists there else it requests the drive emulation to provide the data from local disks to the cache and then returns it back to the host.

In a scenario where R1 local mirrors have failed and are not able to provide the data to any read request from host at the primary side, SRDF emulation will help in serving the read request from the R2 side. SRDF emulation will try to get the data from the R2 devices over the SRDF links and provide it to the host. Host might see a slight performance issue but still it will be able to access the data.


Now the question is that how will we maintain the data integrity and data consistency in SRDF operations and what happens if SRDF link fails etc. I will be discussing about these in my next blog.

Monday, 23 June 2014

SRDF Device States

While performing SRDF operations, SRDF devices (Check my blog for details on various SRDF Devices) may undergo various kind of transitional states like any standard device and SRDF device transitions that are unique to SRDF devices. You can determine the state of the devices after examining the R1 and R2 devices and their mirror states. Enginuity 5874 onwards, you can check the time stamp when this state transition has happened and the cause behind this.

SRDF device states can be determined by a combination of 2 views :

  1. Host interface view - Reflects the SRDF device state as seen by the host
  2. SRDF view - Reflects the SRDF mirror states and internal device state



Host Interface View :
Host interface view reflects the SRDF device states as seen by the hosts which are connected to the device

R1 Device States: R1 device can have any one of the below states

  • Read/Write                                - In this state host will be able to perform read and write operations on the device. This is default R1 device state.
  • Read Only (Write Disabled)      - In this state host will only be able to read from the device but write operations will be disabled for the device.
  • Not Ready                                 - In this state R1 device responds Not Ready to the host for any read and write operations.

Also check Dominos Mode and Data Integrity.

R2 Device States: R2 device can have any one of the below states

  • Read Only (Write Disabled)      - In this state host will only be able to read from the device but write operations will be disabled for the device.
  • Read/Write                                - In this state host will be able to perform read and write operations on the device. We can see this state during recovery or parallel processing from R1 and R2
  • Not Ready                                 - In this state R2 device responds Not Ready to the host for any read and write operations.


Invalid tracks Attributes:

If used with special device attributes,the invalid tracks attributes, the not ready state of the R2 device provides information about the state of SRDF device pair i.e. whether they are usable for business operation or not. Invalid Track attributes can only be set on R2 devices. This attribute tells you that there are invalid tracks which are still not synchronized to R2 devices. If the invalid track attribute is set on R2 devices, Enginuity will make the R2 device Not Ready to the host if the secondary device is not synchronized with R1 device.

In case of a disaster, invalid track attribute helps you in deciding whether to use the R2 devices for recovery or not. If the devices becomes Not Ready due to invalid track attributes, data might not be suitable for business operations. There is option of resetting the Not Ready attribute by using host based software.

SRDF View :
SRDF view reflects the SRDF mirror states and internal device state and also tells whether the device is ready for SRDF operation or not.

R1 Device States: R1 device can have any one of the below states for SRDF operations.

  • Ready     - This condition occurs when the R1 device is able to send the data across SRDF links. It will remain valid even if the local mirror fails or are not ready for the I/O operations as it will not have any impact on the SRDF operations.

  • Not Ready (SRDF mirror Not Ready) - This condition occurs when R1 device is not Not Ready for SRDF operations. It is important to understand that this is an R1 SRDF mirror state and not R1 or R2 device state. This condition occurs when the remote mirroring is halted and updates are not been pushed to R2 device. Invalid tracks keeps accounting on R1 SRDF mirror. When the R2 device's host interface view is Read/Write enabled, R1 will automatically be in "Not Ready" SRDF view state.
R2 Device States: R2 device can have any one of the below states for SRDF operations.

  • Ready    - R2 device is able to receive the updates from R1 device and device is also able to get host based software commands.

  • Not Ready-This state is actually confusing, confuses me every time if I just see the name. In this state actually R2 device can receive updates from R1 device but only restriction will be that it can not receive host based software commands.

  • Link Blocked (lnkBlk)- This state is specific to R2 SRDF mirrors and belong to SRDF R22 devices (Check my blog for details on various SRDF Devices). In this state one of the R2 SRDF mirror can not receive the data from it's associated R1 device.

Various SRDF Devices


Saturday, 21 June 2014

SRDF - Symmetrix Remote Data Facility

SRDF is an EMC remote replication solution for business continuity and disaster recovery solution where you can have an almost real time copy of the data available at the remote site also. This remote site can be geographically located in the same area or anywhere in the world. Geographically located data centres are useful in case of a disaster.

So in a basic definition form you can say that in SRDF we will have a exact copy of the source data at the remote site and this copy will be host independent and will be taken by storage itself. Data will be replicated from the source site to remote site over the specialized networks meant for carrying heavy amount of data.

So question arises why do we need remote replication ? As discussed earlier, if a disaster happens, the data stored might get lost and the applications running on them will be down till the time we restore the situation. So to avoid this kind of situation, we needed a copy of the data from where we can restore everything. Now you might ask, why not use Tape backups or a local copy of the data in place of a remote copy. Idea behind not having only copy of the data as local is to prevent any local disaster kind of situation. While Tape backup is still a solution but should not be used as a single option for data copy, specially for highly critical applications. As we all know restoring from tape is a very time consuming process and can not be a desired solution for application which needs to be available round the clock.


SRDF logically pairs a device or a group of devices from each array and replicates data from one to the other synchronously or asynchronously. An established pair of devices can be split, so that separate hosts can access the same data independently (maybe for backup), and then be resynchronised.

                                             Fig 1 : Typical SRDF Configuration

There are various modes of SRDF operations (Keep checking my Blog for a post on SRDF modes), out of them most popular are SRDF/S and SRDF/A .

In synchronous mode (SRDF/S), the primary array waits until the secondary array has acknowledged each  write before the next write is accepted, ensuring that the replicated copy of the data is always as current as the primary. However, the latency due to propagation increases significantly with distance.

Asynchronous SRDF (SRDF/A) transfers changes made to the secondary array in units called delta sets, which are transferred at defined intervals. Although the remote copy of the data will never be as current as the primary copy, this method can replicate data over considerable distances and with reduced bandwidth requirements and minimal impact on host performance.

Other forms of SRDF integrate with clustered environments and to manage multiple SRDF pairs where replication of multiple devices must be consistent (such as with the data files and log files of a database application).

Now the question arises that how the data flows from one symmetrix box to another remote box, what replication medium does it use and how far can you replicate, will it copy instantly or it's a time taking activity, what will be RTO and RPO for this, what kind of devices can participate in SRDF operations. I will surely be posting details on these topics in my coming blogs. Till then enjoy !

Also check Various SRDF Device States

Friday, 20 June 2014

What is Zero Space Reclamation !


Many application have the tendency to write zeroes to the free space as part of the standard initialization, allocation or migration processes. Depending on the environment, this might be a waste of space for the customers. There is a possibility of reclaiming this space and the process is called "Zero Space Reclamation" in EMC terminology.

Generally speaking when a file is deleted from a file system, it is removed from the file system from being referenced and becomes inaccessible for the file system, however the data on the disk is still non zero and contains the original disk contents. Over time, as the files are continuously added and deleted there might be chances that most of the file system might have non zero data though the file system might report that lot of space is available to use. When these devices are migrated, depending on the migration tool used, it is possible that this non zero data also gets migrated to the new environment which is actually waste of resources.

Generally thick devices used by many applications will have the pre-allocated zero data chunks. Suppose if you have a 40 GB device which is only 20 GB used and you are going to migrate it to a thin device using a migration tool which copies pre-allocated zero data also, now after migration you will notice that the actual usage of the thin device from the thin pool is way more than 20 GB though it should only use 20 GB as it's a thin device. The reason for this is that during migration, extra pre-allocated zero data has also been copied along with any "deleted non zero data" which together accounts for this inflated size. This is actually waste of resources for the user and here "zero space reclamation" is useful.

With Enginuity 5875.135.91 onwards, SRDF supports zero space reclamation while replicating the data from thick LUN to thin LUN. Earlier supported only for FBA devices but with latest releases CKD devices are also supported). Zero space reclamation is an Enginuity feature which allows you to remotely mirror a thick SRDF device with a thin SRDF device while avoiding mirroring the pre allocated zero data chunks that may be associated with a thick SRDF device.


When we start space reclamation process, it starts a back end disk adaptor task to examine the allocated thin device extents for the specified thin devices. A thin device extent is 768 KB (or 12 tracks) in size and is the default unit of the storage at which data is stored in thin devices. For each extent all the 12 tracks will be brought in to symmetrix cache and will be examined if they all contained all zero data. If the entire extent contains all zero data, the extent will be de-allocated and added back to the pool, which will make it available for the next extent allocation    . Should any portion of the extent contains some data, none of the extent is reclaimed.

You can trigger the Zero space reclamation using SYMCLI or via Unisphere GUI. You can run a "symconfigure" command with "start reclaim" command line to start the process. Below example will give you clarity.

root# symconfigure -sid 1234 -cmd "start reclaim on tdev 0345;" commit

Once initiated, the status of the thin device will change from "bound" to " reclaiming". You can run the below command to see the status.

root# symcfg -sid 1234 show -thin -pool FC -detail -GB

It is a very simple process using Unisphere GUI. Just select the tdev, right click and select "start allocate/free/reclaim" and then select start reclaim.

Thursday, 19 June 2014

Data Migration and Why is it Needed ?

Data Migration:

In simple definition, data migration means moving the data from point A to point B. These points can be any storage arrays, JBODs, internal disks etc. 

Data migration is the process of transferring data between data storage systems, data formats or computer systems. A data migration project is usually undertaken to replace or upgrade servers or storage equipment, for a website consolidation, to conduct server maintenance or to relocate a data center.

Though Data Migration is a quite complex procedure but below diagram explains it in a simpler way.


Reasons to move the data :

  • Data Growth: We all might have seen the exponential growth of data over the years and at one particular point of time the array in your environment might not be able to hold the complete data of your company. In that case you might have to replace your smaller array with a larger capacity array and in those scenarios data migration will come into picture.
  • Technical Refresh: The need to migrate due to ageing technology is called technical refresh and it's one of the main reasons to migrate the data for most of the companies.
  • Performance Improvements: Performance improvement can be achieved by many other ways but if nothing works out and it's really required to replace the old array with a better performing latest array then also a data migration will come into picture.
  • Information Life Cycle Management: All information is not created equal and should use most appropriate storage is a key element of ILM. The fact that the priority/value of the information increases/decreases over the time, leading to the need to migrate the data after it's original prioritization and placement. For example data might have to be moved to lower or higher tier or array.
  • Risk Reduction: To lower the risk of data availability and data loss conditions, data migration to a more highly available storage platform leads towards mitigating this risk.
  • Consolidation, localization and regulatory compliance: These are more of business reasons to migrate the data rather than technical. Consolidation of storage due to a merger, centralization or data center migrations needs data also to be migrated accordingly. Localization needs for distributed operations and is a reverse of centralization. Regulatory compliance may require you to keep the archived data accessible for a certain period of time.
  • Complexity and transformational migrations: To optimize the infrastructure, simplify management, reduce risks, lower cost and effectively transform the entire IT infrastructure may lead to data migration.

Wednesday, 18 June 2014

SYMCLI CHEAT SHEET

SYMMASKDB
=========

Initializing the VCMDB
# symmaskdb -sid 5678 init -f vcmdb.db

Backing up the VCMDB – backs up the VCMDB database to a file
# symmaskdb -sid 5678 -file <backup_filename> backup

Restoring the VCMDB – restores a database file
# symmaskdb -sid 5678 -file <restore_filename> restore

Refresh the database (Not always needed)
# symmask -sid 5678 refresh

Check devices masking to hosts
# symmaskdb -sid 5678 -dev 7D5,03A4:03A7 list assignment (DMX)
# symaccess -sid 5678 -dev 7D5,03A4:03A7 list assignment (VMAX)

Listing devices which are masked to given host
# symmaskdb –sid 5678 list database
# symmaskdb -sid 5678 list capacity -host hostA (DMX)
# symmaskdb -sid 5678 list capacity -host hostA -dir 07a -p 1 (DMX)
# symmaskdb -sid 5678 list database -wwn 10000000c868bd8e
# symmaskdb -sid 5678 list database -wwn 10000000c9598f73 -dir 07a -p 1
# symmaskdb -sid 5678 list devs -wwn 10000000c9598f7

Check which devices are masked to given FA port
# symmaskdb -sid 5678 list database -dir 07A -p 0

=====================================================================================
SYMDEV
======

Listing all devices within the array
# symdev list
# symdev -v list (will take a long time to run because of the sheer amount of output)

Checking Device details
# symdev -sid 5678 show 0200

Checking device mapping to any ports (Please See: ???:? indicates device is not mapped to any port; ***:* indicates it is mapped to more than one port)
# symdev -sid 5678 show 214 (single device)
# symdev -sid 5678 -range 6F1:6F5 list (multiple devices)
# symdev -sid 5678 -range 6F1:6F5 list -multiport (multiple devices mapped to multiple ports – works with range only – could specify same device twice in the range if needed for a single device – for example 513:513)

List which devices are mapped to FA ports
# symdev -sid 5678 list -sa 08c -p 0

Listing devices (hypers/metas) which are not mapped to any ports
# symdev -sid 5678 -noport list

How to write disable several devices

# cat devs
0205
0206
# symdev -sid 5678 write_disable -file devs -SA 9c -p 0
# symdev -sid 5678 write_disable -file devs

(This will disable devs on all ports)

How to write disable only a single device

# symdev -sid 5678 write_disable 0123 -SA all
# symdev -sid 5678 write_disable 0123 -SA 14A -p 0

How to change the status of device to “not_ready”

# cat devs
0205
0206


# symdev -sid 5678 not_ready -file devs -SA 9c -p 0
# symdev -sid 5678 not_ready -file devs (This will disable devs on all ports)
# symdev -sid 5678 not_ready 0325 -SA all (This will disable devs on all ports)
# symdev -sid 5678 not_ready 0325 -SA 14A -p 0 (This will disable devs on 3a:0 ports)

================================================================================
SYMDISK
=======
Display the summary information about diskgroups on given array

# symdisk -sid 5678 list -dskgrp_summary -disk_group all
# symdisk -sid 5678 list -dskgrp_summary -disk_group 1
# symdisk -sid 5678 list -dskgrp_summary

Display details about disks in array separated by disk groups
# symdisk -sid 5678 list -by_diskgroup

Display details about disks in given diskgroup
# symdisk -sid 5678 list -disk_group 1

Display the list of disks configured as hotspares in the array
# symdisk -sid 5678 list -hotspare

Display the details of all hotspares in the array
# symdisk -sid 5678 list -v -spare_info

Display the list of failed drives in the array
# symdisk -sid 5678 list -failed

Display list of all disks installed in the array

# symdisk -sid 5678 list
Ident  Symb Int TID Vendor     Type       Hypr   Total       Free      Actual
------ ---- --- --- ---------- ---------- ---- ---------- ---------- ----------
DF-5A  05A    C   0 SATAHGST   C02THJK      24     953870      32940    1823565


Disk ID would be 5A:C0

Display the detailed information such as speed/make/model/diskgroup/hypers on disk
# symdisk -sid 5678 show 5A:C0

Display the gaps (available space) on given disk
# symdisk -sid 5678 show 5A:C0 -gaps

Display only the gaps (available space) on given disk
# symdisk -sid 5678 show 5A:C0 -gaps_only

How to find out datadevs in degraded status
# symdev -sid 5678 -datadev -service_state notnormal list

================================================================================
SYMCFG
======

List of FA ports/WWN on given array
# symcfg -sid 5678 -FA all list
# symcfg -sid 5678 -FA 8C list

List detailed information on a FA port
# symcfg –sid 5678 -FA 11E list -v

Find out the failed director (front end FA and disk DA)
# symcfg -sid 5678 list -dir all => get the FA or DA number from here
# symcfg -sid 5678 list -fa 5g -v | egrep "Negotiated Speed (GB/Second)" => speed should be a number
# symcfg -sid 5678 list -da 5d -v | egrep "Director Status"

Display details about array
# symcfg –sid 5678 list -v

List LUN addresses available on a given FA (and also to check which luns are mapped to which devices)
# symcfg –sid 5678 list -address -available -fa 7e –p 0

To find out which lun number is used to map a device for given host
On VMAX # symaccess -sid 5678 show view hostname_vwa (Column “Host Lun” under Storage group name)
On DMX # symmasksdb -sid 5678 list devs -wwn 50005678567890 (Column “HOST” under LUN)

List which applications (symcfg/stpdaemon etc) are running the commands against given array
# symcfg –sid 5678 -application list

Display details (OS/IP/Hardware/hostname) about hosts that are accessing given array
# symcfg –sid 5678 -connections list

Check the consumed capacity of TDEV (space consumed by a TDEV)
# symcfg -sid 5678 list -pool poolname -tdev -detail -gb | grep 1C4B
# symcfg -sid 5678 list -pool poolname -tdev -range 1A11:1A1B -detail -gb

================================================================================

SYMMASK
=======

Set or clear the FC ID lockdown
# symmask set lockdown

Set or clear visibility in masking database so that host should find all devices, even if they are noncontiguous LUNs
# symmask set visibility

Set or clear LUN base/offset skip for noncontiguous LUNs
# symmask set lunoffset

Set the record in the database to hold information on the host type that may differ than the current setting on the corresponding FA
# symmask set heterogeneous

Discover the WWN or iSCSI names of the HBAs on the host and writes the ASCII names to the login history table, if empty
# symmask discover hba

Display list of devices seen by WWN of given HBA for given host

# symmask list hba
# symmask list hba -v

Display WWNs logged into given FA port (displays the login history table contents)
# symmask -sid 5678 list logins (for the whole array)
# symmask -sid 5678 list logins -dir 3a -p 0 (on given port)
# symaccess –sid 5678 list logins –dirport 5e:0 (on VMAX)

How to mask the devices to given WWN (Add a device to the list of devices that a WWN can access – only for DMX)
# symmask -sid 5678 -dir 8a -p 0 -wwn 100000056785678 add devs 200,200:205
# symmask -sid 5678 -dir 9d -p 1 -wwn 100000056785679 add devs 46a:479,4b8 -dynamic_lun (in case dynamic_lun is in use)
# symaccess –sid 5678 –name hostname_sga –type storage add devs 200,200:205 –lun 2c (on VMAX – to map from lun number 2c onwards)

How to unmask the devices from given WWN (Remove a device from the list of devices that a WWN can access)
# symmask -sid 5678 -dir 8a -p 0 -wwn 100000056785678 remove devs 20c,201:203
# symaccess –sid 5678 –name hostname_sga –type storage remove devs 200,200:205 –unmap (on VMAX – removing and unmapping)

Delete the login entries for a host on FA port
# symmask -sid 5678 -wwn 100000056785678 delete -dir 8a -p 0 -login

How to assign an alias name to a wwn logged into a FA port
# symmask -sid 5678 -dir 8a -p 0 -wwn 10000005678567 rename hostname/hba1 (DMX)
# symaccess -sid 5678 rename -wwn 10000005678567 -alias hostname/hba1 (VMAX)

How to refresh masking database after masking the new luns to a WWN or removing the luns from a WWN (refresh WWN related memory tables with the contents of VCMDB)
# symmask -sid 5678 refresh

Turn on the flag for different OSs (D for Solaris; C,SC3,OS2007,SPC2 for Windows/VMWare, SPC2 for linux)

On DMX
# symmask -sid 5678 set hba_flags on D –enable -wwn 10000005678567 -dir 8C -p 1 (DMX)
# symmask –sid 5678 refresh

On VMAX
# symaccess –sid 5678 –type init show hostname_ig -detail
# symaccess -sid 5678 set hba_flags on C,SC3,OS2007,SPC2 -enable –wwn 100000000567856
# symaccess -sid 5678 set hba_flags on C,SC3,OS2007,SPC2 -enable –name hostname_ig –type init

Globally on FA port
set port 1d:0 SCSI_3=enable,SPC2_Protocol_Version=enable,SCSI_Support1=enable;

================================================================================
SYMCONFIGURE
============

Verify that the changes can be made to the array
# symconfigure -sid <sid> -f <filename> verify

Confirm that the syntax is correct
# symconfigure -sid <sid> -f <filename> preview

Confirm that the change is correct
# symconfigure -sid <sid> -f <filename> prepare

Commit the changes to the database
# symconfigure -sid <sid> -f <filename> commit

Create IO Fencing devices for cluster
# symconfigure -sid 5678 -cmd "create dev count=24, size=2184 cyl, emulation=FBA, config=2-Way-Mir disk_group=3;" prepare

Create gatekeepers
# symconfigure -sid 5678 -cmd "create dev count=24, size=6 cyl, emulation=FBA, config=2-Way-Mir disk_group=3;" prepare

Create raid5 (7+1) thick device (when emulation mode is disabled number of cylinders should be halved to achieve the same size)

# symconfigure -sid 5678 -cmd "create dev count=64, size=49604 cyl,
emulation=FBA, data_member_count=7, config=raid-5 disk_group=3;" prepare
# symconfigure -sid 5678 -cmd "create dev count=64, size=24802 cyl,
emulation=FBA, data_member_count=7, config=raid-5 disk_group=3;" prepare

Create new replicated tdevs on local and remote arrays, bind them to pool, srdf pairing relationship between them
# symconfigure -sid 5678 -cmd "create dev count=64, size=24802, emulation=FBA, config=RDF1+TDEV, dynamic_capability=dyn_rdf, remote_config=RDF2+TDEV, ra_group=1, binding to pool=source_rep_pool, remote_pool=target_rep_pool;" prepare

Modify FA Port settings
# symconfigure -sid 5678 –cmd ‘set port1C:1 volume_set_addressing=enabled;’ prepare (this is a setting for HP systems)

How to create metas from hypers
# symconfigure -sid 5678 –cmd ‘form meta from dev 001 config=striped, stripe_size=1920; add dev 002:004 to meta 001;’ prepare

How to dissolve hypers from meta
# symconfigure -sid 5678 –cmd ‘dissolve meta dev 04BE;‘ prepare

How to map the devices to FA port (if devices are not contiguous, then you would need a new line for each device – you can’t use commas to separate them)

# cat map_devs
map dev 0200:0202 to dir 08b:0, starting lun=06e;
map dev 0200:0202 to dir 09b:1, starting lun=06e;

OR

map dev 0200 to dir 08b:0, lun=06e;
map dev 0200 to dir 09b:1, lun=06e;
map dev 0201 to dir 08b:0, lun=06f;
map dev 0201 to dir 09b:1, lun=06f;
# symconfigure -sid 5678 -file map_devs prepare

Rule of pair mapping (rule of 17):
On DMXs – Ports are [3,4,7,8,9,10,13,14][a-d]. a to d, b to c with rule of 17 and ports opposed. So 3a:0 will be partnered with 14d:1. 4b:1 will be paired with 13c:0.
On VMAXs – Ports are [5-12][e-h]. e to e, f to f, g to g, h to h with rule of 17. so 5f:0 will be 12f:0, 8g:0 will be 9g:0 and so on.

How to unmap devices from FA ports (if devices are not contiguous, then you would need a new line for each device – you can’t use commas to separate them)

# cat unmap_devs
unmap dev 0200:0204 from dir 07B:1;
unmap dev 0200:0204 from dir 04B:0;
unmap dev 0206 from dir all:all;
# symconfigure -sid 5678 -file unmap_devs prepare

How to set device attribute dyn_rdf on/off
# symconfigure -sid 5678 -cmd 'set dev 1001:1004 attribute=dyn_rdf;' prepare

How to set SCSI3-Persistent Reserve
# symconfigure -sid 5678 -cmd ‘set dev 2500:2502 attribute=SCSI3_persist_reserv;’ prepare

How to reclaim wasted space from a TDEV (Zero Space Reclaim)
# symconfigure -sid 5678 -cmd 'free tdev 1A1B start_cyl=0 end_cyl=last_cyl type=zero;' prepare

================================================================================
SYMDG/SYMLD
===========

Create a device group
# symdg create testdg

Delete a device group
# symdg delete testdg -force (if it has members, use -force)

Display the list of device groups
# symdg list <-sid 5678>

Show the details of a particular device group
# symdg show testdg

Display all the devices in a device group
# symld -g testdg list

Add the devices to a device group
# symld -sid 5678 addall -range 0200:0205 -g testdg (bulk)
# symld -sid 5678 add dev 0200 -g testdg(individual)

Remove the devices from a device group
# symld -sid 5678 rmall -range 0200:0205 -g testdg (bulk)
# symld remove dev DEV001 -g testdg (individual – must be logical name)

How to write_disable disks in a device group
# symld -sid 5678 -g testdg -sa ALL write_disable (disables devices in DG on all FAs)
# symld -sid 5678 -g testdg -sa 9c -p 0 write_disable DEV001 (device number must be DEVXXX and can be obtained by symld -g testdg list)

================================================================================
VMAX
====

Bind TDEV to pool with preallocated storage
# symconfigure -sid 5678 -cmd 'bind tdev 1001 to pool fast_pool preallocate size = 24802 CYL;' prepare
# symconfigure -sid 5678 -cmd 'start allocate on tdev 1001 start_cyl=0 end_cyl=<last_cyl>;' prepare (in case above command fails or to allocate the storage later)

Set the maximum subscription of a pool
# symconfigure -sid 5678 -cmd "set pool fast_pool, type=thin, max_subs_percent=100;" prepare

How to list the views created on the array
# symaccess -sid 5678 list view [–v]

How to display details of a view created
# symaccess -sid 5678 show view hostname_vwa -detail

How to list the groups created on the array
# symaccess -sid 5678 list -type storage|port|initiator

How to display the details of a group created
# symaccess –sid 5678 show hostname_sga -type storage
# symaccess –sid 5678 show hostname_iga -type initiator
# symaccess –sid 5678 show arrayname_5e0_pga -type port

How to create initiator group
# symaccess -sid 5678 create -name hostname_iga -type initiator (just create)
# symaccess -sid 5678 create -name hostname_iga -type initiator -wwn 100000000000001

How to add (or remove) a WWN from a Initiator group
# symaccess -sid 5678 add -name hostname_iga -type initiator -wwn 100000000000001
# symaccess -sid 5678 remove -name hostname_iga -type initiator -wwn 100000000000001

How to create Storage Group and add devs at the same time
# symaccess -sid 5678 create -name hostname_sga -type storage (just create)
# symaccess -sid 5678 create -name hostname_sga -type storage devs 1001:1005

How to add (or remove) devices to storage group
# symaccess -sid 5678 add -name hostname_sga -type storage dev 1001
# symaccess -sid 5678 remove -name hostname_sga -type storage dev 1001 -unmap

How to create a port group
# symaccess -sid 5678 create -name arrayname_5e0_pga -type port (just create)
# symaccess -sid 5678 create -name arrayname_5e0_pga -type port -dirport 5E:0

How to add (or remove) director ports to a port group
# symaccess -sid 5678 add -name arrayname_5e0_pga -type port -dirport 5E:0
# symaccess -sid 5678 remove -name arrayname_5e0_pga -type port -dirport 5E:0

How to create a view
# symaccess -sid 5678 create view -name hostname_vwa -sg hostname_sga –ig hostname_iga -pg arrayname_5e0_pga

How to rename a group
# symaccess -sid 5678 rename -name hostname_sga -type storage -new_name newhost_sga

How to rename a view
# symaccess -sid 5678 rename view -name hostname_vwa -new_name newviewname_vwa

How to delete a view (unmapping the devs at the same time)
# symaccess -sid 5678 delete view -name hostname_vwa -unmap

How to delete a initiator group
# symaccess -sid 5678 delete -name hostname_iga -type initiator
Error: Cannot perform the requested operation because the group is not empty
# symaccess -sid 5678 delete -name hostname_iga -type initiator -nop -force

How to delete a storage group

# symaccess -sid 5678 delete -name hostname_sga -type storage
Error: Cannot perform the requested operation because the group is not empty
# symaccess -sid 5678 delete -name hostname_sga -type storage -nop -force

How to replace a wwn in a initiator group
# symaccess -sid 5678 replace -wwn 100000000000001 -new_wwn 2000000000000002

How to rename (alias) for a given WWN
# symaccess -sid 5678 rename -wwn 100000000000001 -alias hostname/hba_a (this can be done from local symcli server only)

How to check which storage group a device belongs to
# symaccess –sid 5678 list -type storage –dev 1001

How to check which initiator group a wwn belongs to
# symaccess –sid 5678 list -type initiator -wwn 100000000000001

How to check which port group a FA port belongs to
# symaccess –sid 5678 list -type port -dirport 7f:1

How to list all tdevs defined on the array
# symdev -sid 5678 -tdev list

How to check the consumed capacity of a given tdev

# symcfg -sid 5678 list -pool poolname -tdev -detail -gb | grep 1C4B
# symcfg -sid 5678 list -pool poolname -tdev -range 1A11:1A1B -detail -gb

How to list all datadevs defined on the array
# symdev –sid 5678 –datadev list

How to list all tdevs not bound to any pool
# symdev -sid 5678 -tdev list -unbound

How to list all tdevs bound to given pool
# symcfg -sid 5678 -tdev -pool testpool

How to list all tdevs not assigned to any host

# symdev -sid 5678 -tdev list -noport -unbound
# symdev -sid 5678 -tdev list -noport -bound

How to check details of a pool (or list all tdevs bound to a pool or list all datadevs constituting the pool)
# symcfg -sid 5678 show -pool fast_pool -thin -detail

How to unbind a tdev from pool
# symconfigure -sid 5678 -cmd 'unbind tdev 1001 from pool fast_pool;' prepare
# symconfigure -sid 5678 -cmd 'unbind tdev 1001:1004 from pool fast_pool;’ prepare

How to check the status of device being unbound
# symcfg -sid 5678 show -pool fast_pool -thin -detail | grep Unbind

How to create TDEVs
# symconfigure -sid 5678 -cmd "create dev count=1000, size=32GB, config=TDEV, emulation=fba, dynamic_capability=dyn_rdf, binding to pool=poolname;" prepare

How to create VDEVs
# symconfigure -sid 5678 -cmd "create dev count=24, size=24802, emulation=FBA, config=VDEV;" prepare

How to create metas from TDEVs (TDEVS should not have been bound to any pool)
# symconfigure -sid 5678 -cmd 'form meta from dev 1000, config=STRIPED, stripe_size=1920; add dev 1001:100F to meta 1000;' prepare

How to list the pools created on the array
# symcfg -sid 5678 list -pool -thin -GB

How to rebalance the pool after adding extra datadevs
# symconfigure -sid 5678 -cmd "start balancing on pool fast_pool type=thin;" prepare
# symconfigure -sid 5678 -cmd "stop balancing on pool fast_pool type=thin;" prepare

=====================================================================================
SYMLMF
========

Display the array based license
# symlmf –sid 5678 list -type emclm

Display the host based license
# symlmf list –type host

Display host based and array based licenses that apply to Symm array
# symlmf –sid 5678 list –type sym

Display the state and usage number for all activated licenses on Symm 5678
# symlmf –sid 5678 query –type emclm

Install the array based license
# symlmf –sid 5678 add –type emclm –file <filename> -v

Install the host based license
# symlmf add –type se –license LicenseNumber

Delete the license
# symlmf delete –type se –license <licensename> where <licensename> is FAST, SRDF, SRDf/Async etc

=====================================================================================
SYMINQ & SYMPD
==============
to check the status of all paths
# syminq -fibre hba

List the devs that the host can access from an array (to be run on the target host)
# sympd list
# sympd list

List device ids and symmetrix serial number of devices server can see
# sympinq -pdevfile

=====================================================================================
SYMSTAT
=======
To get the stats for disks
# symstat -sid 5678 -type disk -disk all -i 5 -c 1

To get the stats for FAs – IOPS and KBPS
# symstat -sid 5678 -type port -dir all -i 5 -c 1
# symstat -sid 5678 -type port -dir 5f -i 5 -c 1
=====================================================================================
STORDAEMON
==========
to check the status of daemons
# stordaemon list -all

To show details of a daemon
# stordaemon show <name_of_daemon>

To start a daemon
# stordaemon start <name_of_daemon>

To stop a daemon
# stordaemon shutdown <name_of_daemon>
=====================================================================================

EMC Interview Questions Part 1

  • How many Cache directors, Front-end directors and Back-end directors we can use in DMX-4 ?
Cache Directors = 4 Min to 8 Max.
Front-end Directors = up to 12 max.
Back-end Directors = 6 Min to 8 Max. 

  •  How many hypers can be created in a Single disk in DMX4?
 255 hypers per disk.
  •  What is Vault drives and Hot Spare? 
Vault Drivers: At the time of emergency shutdown of an Array, what ever the data in cache memory will be destaged/saved on temporary drives called vault drives

Hot Spare: At the time of physical drive failure hot spare drives will take place
 


  • Can you explain about Read Hit, Read Miss and Fast Write and Delayed Write?
Read Hit: In a read hit operation, the requested data resides in global memory. The channel director transfers the requested data through the channel interface to the host and updates the global memory directory. Since the data is in global memory, there are no mechanical delays due to seek and latency.
Read Miss: In a read miss operation, the requested data is not in global memory and must be retrieved from a disk device. While the channel director creates space in the global memory, the disk director reads the data from the disk device. The disk director stores the data in global memory and updates the directory table. The channel director then reconnects with the host and transfers the data. because the data is not in global memory, the symmetrix system must search for data on the disk and then transfer it to the channel adding seek and latency times to the operation.

Fast Write A fast write occurs when the percentage of modified data in global memory is less than the fast write threshold. On a host write command, the channel director places the incoming blocks directly into global memory. For fast write operations, the channel director stores the data in global memory and sends a “channel end” and “device end” to the host computer. The disk director then asynchronously destages the data from global memory to the disk device.

Delayed Fast Write: A delayed fast write occurs only when the fast write threshold has been exceeded. That is the percentage of global memory containing modified data is higher than the fast write threshold. If this situation occurs, the symmetrix system disconnects the channel directors from the channels. The disk director then de-stages the data to disk. When sufficient global memory space is available. The channel directors reconnect to their channels and process the fast I/O requires as a fast write. The symmectrix system continues to process read operations during delayed fast writes with sufficient global memory present, this type of global memory operation rarely occurs.


  • What is the maximum hyper size in DMX-4?
Maximum Cylinders = 65520
Maximum Capacity = 59GB or 61425 MB


  • How many members can contain in one Meta?
1 Meta head + 255 members

  • How many LUN IDs available per FA port?
256 LUN IDs are available per FA port, 255 LUN IDs are usable out of 256.

  • What is Dynamic LUN addressing?
This feature will automatically selects and assigns the LUN IDs to the devices while device mapping to the port Instead of manually assigning address to the device while mapping

  • What are the possible device service states and device status states?
Device Service States : Normal, Failed and Degraded
Device Status States : Ready, Not Ready and Write Disable.


  • How do you reserve the devices?
symconfigure -sid XXXX -f createdev.cmd -expire expiration date-owner myself -comment "this devices are reserved for SRDF activity" reserve

  • How do you create the Disk Groups
We can not create Disk Groups, It should be done by changing BIN file by CE.
We can rename the existing disk groups.
Example: symconfigure -sid 207 -cmd “set disk_group 4 disk_group_name = flash_dsks;” -v -nop commit


  • How do you check the free space by Disk group and Array as whole?
By Disk Group : Symdisk -sid XXXX list -by_diskgroup
Array as whole : Symconfigure -sid XXXX list -freespace

  • How do you check the total assigned devices to a particular Host?
Symmaskdb -sid XXXX list devs -wwn "host hba wwn"
  • How do you check the total allocated storage of a particular Host?
symmaskdb -sid XXXX list capacity -Host hostname