Permanent Sparing Process

Permanent Sparing:

 

An automated, self healing process that runs on the service processor to permanently replace a failing drive with a spare drive through a configuration change. Permanent Sparing can also be thought of as a “permanent replacement.”

 

Direct Sparing:

An automated, self healing process that runs on the service processor that invokes a spare drive and adds it as another member of the RAID group. The failing drive is removed when the copy process is finished.

 

Spare drive:

A physical drive that is not user accessible and is reserved for use by Replacement sparing in the event of a drive failure

 

Permanent Sparing Process

When Symmetrix Enginuity detects that a drive is about to fail, the Permanent Sparing

process begins.It looks for a spare drive of the same block size, capacity, and speed

in a good location to permanently replace the failing drive by means of a configuration change. Permanent Sparing is used in:

Enginuity 5874

Enginuity 5875

Enginuity 5876 with RAID 6(14+2)

 

The Permanent Sparing process identifies a good spare location using the following

rules:

No RAID 1 mirror or RAID 5 group member on the same loop

No more than two members of the same RAID 6 group on the same loop

Note:

If the process cannot identify a spare in a good location, or if the Permanent Sparing process cannot begin for any reason, the system calls home to the EMC Customer Support Center to inform that immediate replacement is required. When a suitable spare is identified, the Permanent Sparing process loads a new configuration file in which all the logical volumes initially configured on the failing drive are now configured on the selected spare (new) drive. The configuration change typically takes a few seconds to complete, during which time the configuration is locked. Data is then rebuilt onto the new drive. The Symmetrix system continues to process host I/O requests at the highest priority to minimize any effects on performance. The configuration is not locked during the rebuild process. The failed drive becomes a Not Ready spare in the spare pool and can be replaced at a later date. Multiple spares are planned for each array to be available for subsequent drive failures. The Not Ready state prevents the drive from being introduced onto the back-end Fibre Channel loop in the event of system power cycle, system IML, or disk director IML. The Not Ready setting will be removed once the drive is physically

replaced, at which time it will again become an available spare.

 

EMC VMAX – Solutions Enabler Versions Required For Enginuity Levels

To confirm that your Solutions Enabler version is supported for the level of Enginuity running on your Symmetrix VMAX please reference the following Knowledge Base support.emc.com/kb/52982
The following table details the Minimum required Solutions Enabler version required:

Note: Check EMC Online Support for the latest Solutions Enabler version and release notes.

calculate free space in veritas volume.

# vxdg list
NAME         STATE                ID
ossctedbdg   enabled,cds          1313610087.29.ossdhcp01

Method 1:

# vxdg -g ossctedbdg free

DISK         DEVICE       TAG            OFFSET    LENGTH   FLAGS
ossctedbdg32 xp24k0_08d4 xp24k0_08d4      10401792  256      -
ossctedbdg33 xp24k0_08d5 xp24k0_08d5      10401792  256      -
ossctedbdg34 xp24k0_08d6 xp24k0_08d6      10401792  256      -
ossctedbdg35 xp24k0_08d7 xp24k0_08d7      10401792  256      -
ossctedbdg36 xp24k0_08d8 xp24k0_08d8      10401792  256      -
ossctedbdg43 xp24k0_08ba xp24k0_08ba      419346432 256      -
ossctedbdg213 xp24k0_0081  xp24k0_0081  104943872 942862992  -


Lenth*512/(1024*1024) = value in MB

in this case :  942862992*512/1024*1024=460383

 
Method 2:

# vxassist -g ossctedbdg maxsize
Maximum volume size: 942864384 (460383Mb)

EMC VMAX Online TDEV (striped) Meta expansion (courtesey Brian)

The following took quite a bit of trial and error on my part, when EMC setup our environment, they turned the Autometa feature on, so when a DEV is created larger then 240gb, it automatically rounded up and created 100gb slices and forms striped METAs. In my understanding, striped meta volumes are not really benificial in a thin pooled environment, that EMC best practices call for concatenated METAs (as each concatenated slice is striped across the entire thin pool allready)

Unfortunatly we were already stuck in this config and hundreds of our META volumes were setup as striped. It is far easier to expand a concatenated META (concatenated METAs also dont have hyper size restrictions) I could find no easy document listing how to expand a striped meta that was a TDEV and the only other blog online I found mentioned expanding a standard Device and not a TDEV. So here you go!

         UPDATE: I've since learned from EMC that the advantage of having striped metas in a thin pool environment provides more I/O "hooks" (as our SE put it) on the front end to the host and can increase performance for high I/O intensive applications when compared to a standard TDEV or a concatenated meta.

1. For a protected (online) expansion, create a BCV+TDEV Meta the same size as the TDEV Meta you wish to expand, with the same size hypers.

· Use the "configure" command to create a copy of the original Meta's configuration (Members and hyper size and bind location which you will have to unbind from)

· symconfigure -cmd "configure 1 devices copying dev xxxx(orig meta) overriding config=BCV+TDEV;" prep

· Unbind from thin pool then bind fully allocated and with persistent flag on :::::IMPORTANT:::::

· symconfigure -cmd "unbind tdev xxxx(new BCV+TDEV meta head) from pool <thin_pool>;" prep

· symconfigure -cmd "bind tdev xxxx(new BCV+TDEV meta head) to pool <thin_pool> PREALLOCATE SIZE = ALL, allocate_type = persistent;" prep

· This will take some time (10 to 15 min depending on size) as it needs to pre-allocate all the space to the pool.  Wait for pre-allocation to complete before continuing 

Note: If you manually create the TDEV meta to be used as a BCV, you can use this to convert it: symconfigure -cmd "convert dev xxxx to BCV+TDEV;" prep

2. Create new Meta members for the original META, the same size as existing members (in cylinders), with the number to achieve amount needed for expansion (Leave them unbound)

· symconfigure -cmd "create dev count=4, size=xxxxxx,config=TDEV, emulation=fba;" commit

· Do not bind

3. Then finially run this command to expand the original META, using the BCV Meta created in step 1:

· symconfigure -cmd "add dev xxxx:xxxx to meta xxxx, protect_data=TRUE, bcv_meta_head=xxxx;" prep

It should run for quite some time, as it copies all data to the BCV, expands the original volume, then copies/re-stripes the data back, including on to the new Meta members. We have seen it bring our back end director utilization to over 80% during the entire expansion, so plan on doing it during a time of low utilization.

After the expansion completes the BCV+TDEV can be cleaned up and deleted


==========================================================================
 Convert all the existing  concatenated metas to striped metas.

Here are the steps to do a conversion:

1)    Create a new meta of the same size and number of members of the meta to be converted:   symconfigure –sid <x> -cmd “create dev count=1 size=<x> gb, emulation=FBA,         config=TDEV;” commit
2)    Bind the new meta to a pool:   symconfigure –sid <x> -cmd “bind tdev <x> to pool <pool name>;” commit
3)    Change the new meta to be a BCV:   symconfigure –sid <x> -cmd “convert dev <x> to bcv+tdev;” commit
4)    Preallocate the new BCV:  symconfigure -sid <x> -cmd "start allocate on tdev <x>, start_cyl=0 end_cyl=last_cyl allocate_type=persistent;" commit
5)    Run the conversion:  symconfigure -sid <x> -cmd "convert meta <x>, config=striped, protect_data=TRUE, bcv_meta_head=<BCV device>;" commit

The last step will take some time;  it took about 30 minutes to convert a 200GB two-member meta when I was testing. 

 

Difference between Fabric Switches and Directors

A director is a class of switch where everything is redundant. Two power supplies, two supervisors, redundant backplane, hot upgrades, etc. So that nothing should bring the director level switch down and cause outage

Port states in Brocade

Differences between port states, reported when execute the switchshow command (online, offline, no light, not sync, etc...). 
I noticed that sometimes a particular is disable and its state show "no light" and other time its state says "not sync". Why?

Online/In_Sync
Normal operating port with a connected device / ISL to other switch. "Online" is the operating state, "In_Sync" the physical state of the receiver/transmitter. (to see the in_sync you have to issue a portshow, switchshow doesn't display it)
 
No_Module 

no gbic/sfp inserted

No_Light 

gbic/sfp inserted, maybe even cable connected, but receiving no light
Disabled/Offline

If you disable a port (via portdisable) its operating state goes to offline. Fabric OS doesn't check offline ports, this means the physical port state is nt display as whatever it was

 If there was a device previously connected it displays as disabled/in_sync, if there was none it displays as disabled/no_light (or no_module). This physical state is only updated when you enable the port again.
swFCPortPhyState: noCard(1), noGbic(2), laserFault(3), noLight(4), noSync(5), inSync(6), portFault(7), diagFault(8), lockRef(9)
swFCPortOpStatus: unknown(0), online(1), offline(2), testing(3), faulty(4)

Port State in switchshow Description

No_Card No interface card present
No_Module No module (SFP or other) present
No_Light Module not receiving light
No_Sync Module receiving light but out of sync
In_Sync Module receiving light and in sync
Laser_Flt Module signaling a laser fault
Port_Flt Port marked faulty
Diag_Flt Port failed diagnostics
Lock_Ref Locking to the reference signal
Testing Running diagnostics
Online Port is up and running

How to find the HOST WWN in BROCADE switch?

fcsw2:admin> nodefind 50:01:43:80:02:a3:00:16
Remote:
Type Pid    COS     PortName                NodeName
N    101501;2,3;50:01:43:80:02:a3:00:16;50:01:43:80:02:a3:00:17;
FC4s: FCP
NodeSymb: [41] "QMH2462 FW:v4.04.09 DVR:v8.02.01-k1-vmw43"
Fabric Port Name: 20:16:00:05:1e:04:0c:df
Permanent Port Name: 20:11:00:24:81:76:9a:62
Device type: NPIV Initiator

Port Index: 21 //same as 15 hex above

Share Area: No
Device Shared in Other AD: No
Redirect: No
Aliases: mat_esxs3 mat_esxs3

fcsw2:admin> fabricshow
Switch ID   Worldwide Name    Enet IP Addr    FC IP Addr      Name
-------------------------------------------------------------------------
4: fffc04 10:00:00:05:1e:34:7d:9b 16.53.144.100 0.0.0.0  >"fcsw2"
16:fffc10 10:00:00:05:1e:04:0c:df 16.53.146.146 0.0.0.0        "shishapangma" 

 FCID is 101501: Domain = 0x10(16), Area = 0x15(21), so we're looking for switch ID 0xfffc10, port 21.

Difference between WWN and WWPN?

A WWPN (world wide port name) is the unique identifier for a fibre channel port where a  WWN (world wide name) the unique identifier for the node itself.

A good example is a dual port HBA.  There will be two WWPN's (one for each port) and only a single WWN for the card itself.

It's actually the firt octet that is changing. For example the portName for my multiport Emulex card is 10000000c9592f6c. The NodeName is 2000000c9592f6c. The "1" is simply changed for "2". This is occuring for AIX Servers and Windows Servers. There is also a minor change between the portName and NodeName for my Clariion SP ports. For example portName shows 500601613021788f and NodeName for the same SP port shows 500601603021788f. The 61 changed to 60.

How to find zone directly from Host WWN in CISCO ?

pacdcmdsa3# show zone member ?
  fcalias  Alias Name
  fcid     FCID
  pwwn     WWN
 
 Example:

pacdcmdsa3# show zone member pwwn 21:00:00:1b:32:0e:32:b7
 pwwn 21:00:00:1b:32:0e:32:b7 vsan 1
  zone pacdcldom0014_clarion_spa0
  zone pacdcldom0014_clarion_spb0

How to Find Brocade switch Model?

pacdcbr5100a08_a:admin> switchshow
switchName: pacdcbr5100a08_a
switchType: 66.1
switchState: Online
switchMode: Native
switchRole: Subordinate

switch type output dispalys gives the model number  with 1.X for the brocade 1000 family,2.x for the
brocade 2800 family.The value x refers to motherboard Revision level
 
Switch Type Switch Name
1 Brocade 1000 Switches
2,6 Brocade 2800 Switch
3 Brocade 2100, 2400 Switches
4 Brocade 20x0, 2010, 2040, 2050 Switches
5 Brocade 22x0, 2210,2240, 2250 Switches
7 Brocade 2000 Switch
9 Brocade 3800 Switch
10 Brocade 12000 Director
12 Brocade 3900 Switch
16 Brocade 3200 Switch
17 Brocade 3800VL
18 Brocade 3000 Switch
21 Brocade 24000 Director
22 Brocade 3016 embedded Blade Switch
23 8Gbit 10-port embedded fabric switch
26 Brocade 3850 Switch
27 Brocade 3250 Switch
29 Brocade 4012 EmbeddedBlade Switch
32 Brocade 4100 Switch
33 Brocade 3014 Switch
34 Brocade 200E Switch
36 Brocade FR4-18i DirectorBlade
37 Brocade 4020 Embedded Blade Switch
38 Brocade 7420 SAN Router
40 Fibre Channel Routing (FCR) Front Domain
41 Fibre Channel Routing (FCR) Xlate Domain
42 Brocade 48000 Director
43 Brocade 4024 Embedded Blade Switch
44 Brocade 4900 Switch
45 Brocade 4016 Embedded Blade Switch
46 Brocade 7500 Switch
47 Brocade FC4-16IP Director Blade
50 Brocade 4GB FC Port Blade
51 Brocade 4018 Embedded Blade Switch
55 Brocade FA4-18i Extension Director Blade
55,2 Brocade 7600 Switch
58 Brocade 5000 Switch
62 Brocade DCX Backbone
63 Brocade 8Gb Backbone Core Fabric Switch
64 Brocade 5300 Switch
66 Brocade 5100 Switch
67 Brocade Encryption Switch
68 Brocade 8Gb 16 FC 2 GigE ports Director Encrypti
on Blade
69 Brocade 5410 Blade
70 Brocade 8GB 10 Port Embedded Fabric Switch
71 Brocade 300 Switch
72 Brocade 5480 Embedded Blade Switch
75 Brocade M5424 Embedded Blade Switch
76,6 Brocade 8000 FCoE Switch
77,3 Brocade DCX-4S
82 Brocade 8Gb 24-port Embedded Blade Switch
83 Brocade 16-FC port,
6-GE port, auto sensing 1, 2, 4 or 8Gbit Switch
86 Brocade 8Gbit 26-port embedded Switch
88 Brocade 10Gb 24 GigE ports DCE Blade
89 Brocade 8Gb 12 FC, 1Gb 10 GigE FCIP Blade, 10Gb
2 GigE ports FCR
***************************************************

TYPE SPEED IBM TYPE IBM MODEL NAME BROCADE MODEL NAME
 
2.x 1 Gbps 3534-1RU Brocade 2010
3.x 1 Gbps 2109-S08 Brocade 2400
6.x 1 Gbps 2109-S16 Brocade 2800
9.x 2 Gbps 2109-F16 Brocade 3800 (Cylon)
10.x 2 Gbps 2109-M12 Brocade 12000 (Ulysses)
12.x 2 Gbps 2109-F32 Brocade 3900 (Terminator)
16.x 2 Gbps 3534-F08 Brocade 3200 (Mojo)
21.x 2 Gbps 2109-M14 Brocade 24000 (Meteor)
22.x 2 Gbps IBM BladeCenter Module Brocade 3016 (Blazer)
26.x 2 Gbps 2005-H16 Brocade 3850 (Dazzler)
27.x 2 Gbps 2005-H08 Brocade 3250 (DazzlerJR)
32.x 4 Gbps 2005-B32 SAN32B-2 Brocade4100 (Pulsar)
34.x 4 Gbps 2005-B16 SAN16B-2 Brocade200E (Stealth)
37.x 4 Gbps IBM BladeCenter module Brocade 4020 (Blazer2)
38.x 2 Gbps 2109-A16 SAN16B-R BrocadeAP7420 (Mars)
42.x 4 Gbps 2109-M48 SAN256B Brocade48000 (Saturn)
43.x 4 Gbps HP BladeCenter Module Brocade 4024
44.x 4 Gbps 2005-B64 SAN64B-2 Brocade4900 (Viking)
46.x 4 Gbps 2005-R18 SAN18B-R Brocade7500 (Sprint)
46.x 4 Gbps 2005-R04 SAN04B-R Brocade7500E (Sprint)
58.x 4 Gbps 2005-B5K SAN32B-3 Brocade5000 (Pulsar2)
62.x 8 Gbps 2499-384 SAN768B BrocadeDCX
64.x 8 Gbps 2498-B80 SAN80B-4 Brocade5300
66.x 8 Gbps 2498-40E SAN40B-4 Express Brocade 5100
66.x 8 Gbps 2498-B40 SAN40B-4 Brocade5100
67.x 8 Gbps 2498-E32 Encryption Switch Brocade Encryption Switch
71.x 8 Gbps 2498-24E SAN24B-4 Express Brocade 300
71.x 8 Gbps 2498-B24 SAN24B-4 Brocade300
73.x 8 Gbps 10 port IBM BladeCenter moduleBrocade 5470 (Blazer3)
73.x 8 Gbps 20 port IBM BladeCenter moduleBrocade 5470 (Blazer3)
76.x CEE 3758-B32 IBM Converged Switch Brocade 8000
77.x 8 Gbps 2499-192 SAN384B BrocadeDCX-4S
83.x 8 Gbps 2498-R06 SAN06B-R Brocade7800
121.x 16 Gbps 2499-416 SAN384B-2 BrocadeDCX8510-4
120.x 16 Gbps 2499-816 SAN384B-4 BrocadeDCX8510-8
109.x 16 Gbps 2499-F48 SAN48B-5 Brocade6510