Visitors

Vblock Setup Validation Failed. MDS is not enabled for NPIV

Environment: EMC Unified Infrastructure Manager/Provisioning – Version 4.1 P 01 (Build 4.1.0.1.598)

All of a sudden I could not provision any more datastores or, on close inspection, perform any changes to the Services.

The usual troubleshooting steps of restarting UIM services (service uim-device-services restart), restarting UIM appliance and Rediscovering Vblock did not help.

I tried to validate Vblock and it failed with the following result:

Vblock validation failed. MDS is enabled for NPIV


 

Rule Description:
NPIV provides a means to assign multiple port IDs to a single N Port. This feature allows multiple applications on the N Port to use different identifiers and allows access control, zoning, and port security to be implemented at the application level. You must globally enable NPIV for all VSANs on the MDS switch to allow the NPIV-enabled applications to use multiple N Port Identifiers.

Failure Description:
The MDS discovered in the Vblock does not have NPIV enabled. This is required as per the checklist in the UIM Deployment Prerequisites document.

Remedy Comment:
Configure NPIV feature on the MDS.


 

This is a weird message considering the switch in question is Ethernet switch, not SAN switch and NPIV validation is not applicable.

I have sent the database backup to UIM development team for analysis. Let’s wait what they come up with.

While we are waiting for the solution, here is a quick workaround:

You can change the Severity level for NPIV MDS validation test from ‘CRITICAL’ to ‘WARNING’ in UIM Configuration Manager:

  1. Open UIM Configuration Cnter: Login to UIM management interface, click on the icon next to Logoff/Exit icon in the top right corner of the screen and launch Configuration Center.
    UIM Configuration Center
    You need Java 7 and… It may take you a couple of attempts to launch it…
  2. In the top menu click on Tools and select Automation Library (or press F3);
  3. Navigate to Library Manager – UIM Devices – Vblock Setup Validation Tests – MDS – NPIV MDS;
  4. Change the severity from ‘CRITICAL’ to ‘WARNING’;
    Change Severity of the Vblock Setup Validation Test
  5. Click Save;
  6. Validate Vblock.

Why Ethernet switch gets validated with the MDS validation test? Is this another EMC UIM/P 4.1 bug?

Networker quick script Mitrend assessment: type: NSR not found

I submitted the output of networkerquick.bat script file to Mitrend for assessment but it got rejected because ‘type: NSR‘ was not found in the output.

This turned out to be an issue with the version of NetWorker server I tried to run the script on. The customer had NetWorker 8.0.1.Build.123 installed which is a DA build and should not be used in a prod environment.

Here is the workaround:

  1. Run nsradmin manually to extract missing ‘p type: NSR‘ data:
    C:>nsradmin -i input > nsradmin.out 2> nsradmin.err
    where:
    input – script input file:

    option hidden:on
    p type: NSR
  2. Add content of nsradmin.out to nwquick_nsradmin.out (created by networkerquick.bat)
  3. Zip the content of nwquick folder and upload it to emc.mitrend.com.

It may not be a common issue but anyway, it is good to know the CLI syntax for running nsradmin script from an input file.

VCE™ Certified Professional Program

VCE have finally released VCE™ Certified Professional Program!

Here is some Copy&Paste from the website:

Program Overview

As data center infrastructures converge, the VCE™ Certified Professional program drives success by validating the real world expertise in the evolution of VCE industry leading technologies.

Data center transformation involves more than just technology change. Ultimately, there is a change in business processes and job roles. The role of the VCE Certified Professionals is to bring clarity and quality to the architecting, deployment, and operation of a converged infrastructure environment.

Vblock™ Systems + VCE Certified Professionals = Data Center Transformation

The VCE Certified Professional Program is designed for experienced IT professionals committed to staying relevant in a converged infrastructure environment. This program validates that IT professionals have bridged the gap from a domain-specific focus to a cross-domain focus.

The VCE Certified Professional framework is in place and comprehensive information is now available. New exams to complete our framework will be announced during 2014 as they become live at Pearson VUE. Be the first in your company to be a VCE Certified Professional!

Design

Deploy

Manage

*VCE™ Certified Converged Infrastructure
Master Design Engineer

(VCE-CIMDE)

*VCE Certified Converged Infrastructure
Master Implementation Engineer

(VCE-CIMIE)

*VCE Certified Converged Infrastructure
Master Operations Engineer

(VCE-CIMOE)

*VCE Certified Converged Infrastructure
Design Engineer

(VCE-CIDE)

VCE Certified Converged Infrastructure
Implementation Engineer
(VCE-CIIE)

*VCE Certified Converged Infrastructure
Operations Engineer

(VCE-CIOE)

VCE Certified Converged Infrastructure
Associate
(VCE-CIA)

*Coming Soon

VCE™ Certified Professional Exams

Use the table below to link to exam descriptions, exam content, blueprints, and supporting courseware and documents.

Tracks, Credentials and Exams listed are available at announcement and into mid-2014. VCE will be adding new Tracks, Credentials and Exams as they become live at Pearson VUE.

Exam #

Track

Specialty

Level

Exam Name

Assessment Exam is available online to all professionals.

VCE™ Certified Converged Infrastructure Associate (CIA)

This is the VCE Vblock™ Systems Foundation Exam prerequisite for multiple specialties

Associate VCE Vblock Systems Foundation Exam Included in the VCE Vblock Systems Foundations course.

210-010

Deploy – VCE Certified Converged Infrastructure Implementation Engineer (CIIE)

Core Implementation and Deployment

Engineer VCE Vblock Systems Implementation and Deployment – Core

210-015

Deploy – VCE Certified Converged Infrastructure Implementation Engineer (CIIE)

Virtualization

Engineer VCE Vblock Systems Implementation and Deployment – Virtualization

210-020

Deploy – VCE Certified Converged Infrastructure Implementation Engineer (CIIE)

Compute

Engineer VCE Vblock Systems Implementation and Deployment – Compute

210-025

Deploy – VCE Certified Converged Infrastructure Implementation Engineer (CIIE)

Network

Engineer VCE Vblock Systems Implementation and Deployment – Network

210-030

Deploy – VCE Certified Converged Infrastructure Implementation Engineer (CIIE)

Storage

Available Mid-2014

Engineer VCE Vblock Systems Implementation and Deployment – Storage

VCE™ Certified Professional Program FAQs

Q. What is the VCE Certified Professional Program?
A. The VCE™ Certified Professional Program enables VCE internal, partner and customer engineers who design, deploy, and manage VCE Vblock Systems to demonstrate their expertise by passing a job-role based, legally defensible exam.

Q. Can I take the exams now?
A. At this point, the VCE CI Associate (VCE-CIA) and VCE CI Implementation Engineer (VCE-CIIE) credentials are available.

Q. When will the other tracks be ready?
A. The VCE Certified Professional Program will be rolled out throughout 2014. The next track to be rolled out will be the Manage Track.

Q. Are there legal agreements or terms and conditions associated with the exams?
A. Yes, before a candidate takes the exam they must sign a Candidate Agreement, which is a legally binding document with VCE terms and conditions, as well as security policies.

Q. What does the candidate receive as a result of being certified?
A. Within three (3) weeks of passing the exams required for the VCE credential, the candidate will receive a welcome email and an official VCE Certified Professional digital logo and use guidelines on how to use it. Candidates may use certification logos, certificates, and other collateral in accordance with the published Logo Guidelines in the Candidate Agreement.

Q. Will my certification expire?
A. Earning a VCE Certified Professional credential is a great achievement. Keeping current demonstrates a candidate’s commitment to career growth, their employer and their customer base. The VCE Certified Professional is also recognized for continued growth with VCE. VCE has a 2-year Recertification Policy. Within that 2-year window, a credential is considered Active. A reminder will be mailed when a VCE Certified Professional is within the six month window of a credential lapse.

Q. What is the cost of an exam?
A. The exam required for VCE-CIA certification is available free of charge and is included in the Vblock Systems Foundations eLearning course. Course registration is required in order to access the exam. However, completing the eLearning course is not a prerequisite. For experienced candidates, there is an option to by-pass the course directly to the exam. The exams required for VCE-CIIE certifications cost $200.00 USD for partners and customers for each attempt. Cost is uplifted or decreased depending on local currency. Pearson VUE accepts payment as outlined on their website.

Q. How do I register for a VCE exam?
A. The exam required for VCE-CIA certification is available free of charge and is included in the Vblock Systems Foundations eLearning course. Course registration is required in order to access the exam. However, completing the eLearning course is not a prerequisite. For experienced candidates, there is an option to by-pass the course directly to the exam. Register for any VCE-CIIE exam at Pearson VUE, VCE’s test delivery partner using an exam voucher. Pearson VUE will administer the exam in a secure, proctored environment.

Q. Can you tell me more about VCE Certified Professional exam vouchers?
A. Exam vouchers for VCE Certified Professional exams are electronic vouchers that may be used as payment at Pearson VUE testing facilities, the sole testing provider of VCE Certified Professional exams. Visit the VCE Voucher Store to purchase your vouchers. VCE Vouchers may be purchased in quantities of 10 or more for a 20% discount.
These vouchers are not redeemable for EMC, VMWare, RSA, ITIL or other certification exams.

Q. Can EMC Training Units or EMC Subscriptions be used to purchase an Exam?
A. No form of EMC currency can be applied. The only way to purchase a VCE exam is with a credit card or VCE issued Voucher.

For all other questions, contact VCEcertification@vce.com

EMC UIM/P upgrade from version 4.0 to 4.1 may fail.

Please follow ‘HOW TO: Upgrade EMC UIM/P‘ blog post for instructions how to upgrade UIM/P.

When performing UIM/P upgrade from version 4.0 to 4.1 it may fail with the following error message:

Last Install: Failed to install updates (Error while running pre-install scripts)

EMC UIM Failed to install updates (Error while running pre-install scripts)

If you check /opt/ionix-uim/logs/appliance-update.log you will see the following:

Version – 4.0.0.2.359
Description –
Unified Infrastructure Manager/Provisioning

04/06/2014 11:31:19 9440 main:UA_CLI.cpp:187 (updatecli) [INFO] Starting Install
04/06/2014 11:31:19 9440 updateStatus:UA_CLI.cpp:45 (updatecli) [INFO] Update status: Starting Install
04/06/2014 11:31:19 9440 updateStatus:UA_CLI.cpp:45 (updatecli) [INFO] Update status: Running pre-install scripts
04/06/2014 11:31:19 9440 executeFile:UA_CLI.cpp:86 (updatecli) [INFO] Running /opt/vmware/var/lib/vami/update/data/job/8/pre_install '4.0.0.359' '4.1.0.598'
Installing update from version 4.0.0.359 to version 4.1.0.598
Checking the vendorUUI ...
Checking the productRID ...
Checking vendorUUID and productRID completed successfully...
Vami needs to be updated.

Run the following commands:
cd /opt/ADG/adg_vami25
./InstallVami25Package.sh

04/06/2014 11:31:20 9440 executeFile:UA_CLI.cpp:92 (updatecli) [ERROR] Failed with exit code 256
04/06/2014 11:31:20 9440 updateStatus:UA_CLI.cpp:45 (updatecli) [INFO] Update status: Running post-install scripts
04/06/2014 11:31:20 9440 executeFile:UA_CLI.cpp:86 (updatecli) [INFO] Running /opt/vmware/var/lib/vami/update/data/job/8/post_install '4.0.0.359' '4.1.0.598' 1
Failed with status of 1 while installing version 4.1.0.598
VM version is still 4.0.0.359

It clearly says that VMware Studio Virtual Appliance Management Infrastructure (VAMI) needs to be updated to version 2.5.

To update VMware Studio VAMI:

  • go to /opt/ADG/adg_vami25
  • and execute install script:
UIM-DC1:/opt/ADG/adg_vami25 #
./InstallVami25Package.sh
---------------------------------------
Checking the current vami version before update
Creating logfile: /opt/vmware/var/log/vami/InstallVami25Package.log
Checking the 2.1 vami packages
Checking the 2.5 vami packages
Checking the file: vami_ovf_process
Un-install the old vami packages:
Install the new 2.5 vami packages:
insserv: can not stat(emc-homebase-serverdb)
insserv: can not stat(emc-homebase-server)
insserv: can not stat(emc-homebase-serverdb)
insserv: can not stat(emc-homebase-server)
/opt/vmware/bin/sfcbrepos: line 182: 9572 Segmentation fault (core dumped) sfcbinst2mof -n $namespace -c $instfile -o $instmigfile -r $registrationdir/repository.previous/ -g ${DESTDIR}/opt/vmware/etc/sfcb/sfcb.cfg 2> /dev/null
/opt/vmware/bin/sfcbrepos: line 182: 9573 Segmentation fault (core dumped) sfcbinst2mof -n $namespace -c $instfile -o $instmigfile -r $registrationdir/repository.previous/ -g ${DESTDIR}/opt/vmware/etc/sfcb/sfcb.cfg 2> /dev/null
Replace the file: vami_ovf_process
Adding vami group
Start up the 2.5 vami vami-fscb services
Start up the 2.5 vami vami-lighttp services
vami package updated successfully

Go back to the upgrade page and start again. This time the upgrade should finish successfully.

(c) Zbigniew ‘Ziggy’ Misiak

EMC RecoverPoint & VMware SRM/SRA compatibility matrix

I am currently upgrading VMware vSphere infrastructure from 5.0 Update 2 to 5.5 U1. As part of the upgrade, I need to upgrade VMware Site Recovery Manager (SRM) 5.0.1 to 5.5.1 which is quite straight forward. Check out this and this articles.

But what about EMC RecoverPoint? We are currently running RecoverPoint 3.5 SP2 (p74) and would like to upgrade to 4.0 SP2 P1 (m29).

There are two places where you can get the EMC RecoverPoint & VMware SRM/SRA compatibility matrix:

  • VMware Compatibility Guide (SRM specific)
    N.B. You should select SRA or Replication Adapter to get SRA 2.2, or 2.1 and 2.0
  • EMC Simple Support Matrix:
    • RecoverPoint 3.5 (link)
    • RecoverPoint 4.0 (link)

Here is combined the VMware SRA / SRM & EMC RecoverPoint compatibility matrix:

Partner  Name SRA Name EMC RecoverPoint SRM Version(s)
EMC EMC RecoverPoint Storage Replication Adapter 2.0 EMC RecoverPoint 3.5 and 3.5.1.0 SRM 5.0 Update 3, SRM 5.0 Update 2, SRM 5.0 Update 1, SRM 5.0
EMC EMC RecoverPoint Storage Replication Adapter 2.1 EMC RecoverPoint 3.5.1 SRM 5.1 Update2, SRM 5.1 Update1, SRM 5.1
EMC EMC RecoverPoint SRA 2.2 ECM RecoverPoint 4.0 SP1 SRM 5.5 Update1, SRM 5.5, SRM 5.1 Update2, SRM 5.1 Update1, SRM 5.1

It is not clear what is the correct upgrade path for VMware SRM/SRA & EMC RecoverPoint though…

 

RecoverPoint_Storage_Replication_Adapter_2.X_ReleaseNotes.pdf sheds some light on supported RecoverPoint versions:

Page 3:

RecoverPoint SRA 2.0 supports VMware vCenter Site Recovery Manager Server 5.0 and later with RecoverPoint 3.4 SP1 and later.

RecoverPoint SRA 2.1 is a 64-bit application that supports VMware vCenter Site Recovery Manager Server 5.1 and later with RecoverPoint 3.5 SP1 and later.

Site Recovery Manager 4.X is not supported with RecoverPoint 4.0 and later.

Page 4:

Versions of Storage Replication Adapter and supporting software

 

EMC RecoverPoint Simple Support Matrix:

Page 3:

RP+VMware+Interop

 

Still confused? The following screenshot was provided to me by EMC Support to confirm ALL possible supported combinations of VMware SRM/SRA/vCenter and RecoverPoint:
RecoverPoint Support Matrix

I have also been advised to upgrade RecoverPoint Appliances first and then upgrade SRA and SRM.
Link to the EMC Support community discussion: EMC Support Community > Replication Support Forum > Discussions  > RecoverPoint 4 and SRA version

Hope this will help.

BTW, RecoverPoint/SE is now customer upgradable!!! So, still tuned, RecoverPoint Upgrade blog post is on the way!

UPDATE:
I have upgraded RecoverPoint from 3.5 SP1 to 4.0 SP2 P1 (m29) and can confirm that VMware SRM 5.0 U3 and SRA 2.0 are compatible. Everything is working as it should.

EMC UIM/P cannot start vCenter synchronization

Environment: EMC UIM 4.0 upgraded to UIM 4.1

You may get the following error messages when trying to start vCenter synchronization:

  • * Error initiating synchronization.
    EMC UIM - Cannot Sync vCenter -1
  • viu.error: key not found
    EMC UIM - Cannot Sync vCenter -2

This happens because UIM/P 4.1 doesn’t like special characters like ‘ & ‘ and ‘ ! ‘ etc. in neither Service Offering name nor in Service description.

EMC UIM - Supported Characters

The resolution depends on where you have unwanted special character.

If it’s in the Description field only:

  • Select your Service;
  • Click on edit;
  • On general tab click edit and modify description;
  • Click on Save, click on Close and retry synchronization.

If it’s in Service Offering name – use SAU or contact VCE  Support:

  • Rediscover Vblock;
  • Perform full database backup using /opt/ionix-uim/tools/backup.pl script;
  • Make a note of current vCenter settings;
  • Unmanage Service using SAU;
  • Mark old Service Offering unavailable and delete it;
  • Re-adopt Service – make sure you don’t not use special characters;
  • Put back vCenter settings to your service;
  • Retry synchronization.

(c) Zbigniew ‘Ziggy’ Misiak

EMC RecoverPoint upgrade fails: Unable to distribute ISO file.

EMC RecoverPoint 3.5.1 to 4.0 P1 upgrade fails.

During RecoverPoint “Prepare for Upgrade” wizard the RP upgrade ISO file were uploaded to the RPA. The problem was that the file name did not contain the MD5 checksum in the file name, i.e. “rel4.0.SP2.P1_m.29_release_emc.iso“.

Note from EMC RecoverPointSE Upgrade Guide:

IMPORTANT
For all options, the ISO file name must be the exact same name as the ISO image that you downloaded or obtained using other means. Renaming the file may prevent the upgrade from working properly.

  • rel4.0.SP2.P1_m.29_release_emc.iso – incorrect file name
  • rel4.0.SP2.P1_m.29_release_emc_md5_c172e40591efb211daaf9ec1e52d9e87.iso – correct file name

When we tried to upload the correct file to RPAs, we go the following error message:

“Unable to distribute ISO file. RPA already contains another ISO file, please contact Customer Support.”

RecoverPoint - Unable to distribute ISO file

How can you remove upgrade ISO image from a RecoverPoint Appliance?

  1. Login to each RPA as boxmgmt (default password boxmgmt);
  2. [2]Setup> [8]Advanced options> [6]System internal operations> [1]Run script
  3. Run the following signed script:
    NzUyZWZhNDAyNTdiYTJhMmMyODgxYjcxN2UxOTVhNjcKdW5saW1pdGVkCm5vdF9yZXN0cmljdGVk
    ClRoZSBpZCBvZiB0aGUgc2NyaXB0IGlzOjI0NjU4ClJlbW92ZSBJU08gZmlsZXMKQnJpYW4KIyEv
    YmluL3NoIAojSVNPIFNjcmlwdAojUmVtb3ZlcyBJU08gZmlsZXMKcm0gL2hvbWUva29zL2lzby8q
    Lmlzbwo=
    #
  4. Repeat the procedure on another RPA.
login as: boxmgmt
Using keyboard-interactive authentication.
Password: XXXXXX

** Main Menu **
[1] Installation
[2] Setup
[3] Diagnostics
[4] Cluster operations
[5] Shutdown / Reboot operations
[Q] Quit
DC2 RPA1: 2

** Setup **
[1] Modify settings
[2] Configure repository volume
[3] Get remote settings
[4] Retrieve previous settings
[5] Apply settings
[6] View settings
[7] Reset settings changes
[8] Advanced options
[9] Console configuration
[B] Back
[Q] Quit
DC2 RPA1: 8

** Advanced options **
[1] Clear reservation
[2] Unregister reservation
[3] Clear persistent reservation
[4] Reboot Regulation
[5] Storage device customizations
[6] System internal operations
[7] Security options
[8] Register with ESRS gateway
[9] Set time via NTP
[10] Enable/Disable iperf server
[11] Enable/Disable logging of incoming FC headers
[12] iDRAC support
[B] Back
[Q] Quit
DC2 RPA1: 6

** System internal operations **
[1] Run script
[2] Run internal command
[3] Appliance config params
[B] Back
[Q] Quit
DC2 RPA1: 1
Enter script:
NzUyZWZhNDAyNTdiYTJhMmMyODgxYjcxN2UxOTVhNjcKdW5saW1pdGVkCm5vdF9yZXN0cmljdGVk
ClRoZSBpZCBvZiB0aGUgc2NyaXB0IGlzOjI0NjU4ClJlbW92ZSBJU08gZmlsZXMKQnJpYW4KIyEv
YmluL3NoIAojSVNPIFNjcmlwdAojUmVtb3ZlcyBJU08gZmlsZXMKcm0gL2hvbWUva29zL2lzby8q
Lmlzbwo=
#
Enter your name: admin

Script ran successfully

The script is:
#!/bin/sh
#ISO Script
#Removes ISO files
rm /home/kos/iso/*.iso

The output is:

BANG! And the ISO is gone:

RecoverPoint - Downloading ISO to RPA

Hope this will help.

UIM/P with VMAX – unable to expand striped meta volumes

Currently UIM/P is unable to expand striped metavolumes on EMC VMAX.

If you try to expand stripped metavolume, you get the following error message:

Why is it not supported by UIM? Simple answer is because it’s quite complicated procedure. But nevertheless it is supported by EMC. Expanding striped metavolume requires the use of a Business Continuance Volumes (BCV).
EMC Business Continuance Volumes are special devices used with EMC TimeFinder® software. Regardless, expanding a striped metavolume with a BCV does not require a TimeFinder license.
You can expand device using BCV by following ‘Striped metavolume expansion‘ procedure described in ‘Using EMC Symmetrix Storage in VMware vSphere Environments‘ – see page 59. The corresponding VMFS datastore will need to be manually extended. You will also need to sync the changes with UIM/P database using UIM Service Adoption Utility (SAU).

Is it worth the hassle?
If you have enough space to create a new device with desired size, the answer is no!
You can provision a new correctly sized datastore, storage vMotion all VMs from the old datastore and then decommission it.

If you care about naming convention and you wish to have new datastore name the same as old one you can simply after decommissioning rename device identifier on VMAX, rename datastore in vSphere, rediscover Vblock in UIM/P and use SAU to update UIM/P database to have desired names there.

Also note that if using Solutions Enabler 7.1 and Enginuity 5874 and earlier, bound striped thin metavolumes cannot be expanded online, they must be unbound, unmapped and unmasked. Since this process will cause complete destruction of the data stored on the metavolume, any data on the striped thin metavolume must be migrated off before attempting to expand it in order to prevent the data loss. Beginning with Solutions Enabler 7.2 and Enginuity 5875, a striped thin metavolume can be expanded with no impact to host access or to the data stored on the metavolume with the assistance of a BCV.

(c) Zbigniew ‘Ziggy’ Misiak

Unable to create VCAC database

I was having trouble creating database on vpostgres when tried to create VCAC database.

When logged in as postgres user I typed psql at the prompt and was getting the following error:

-bash: psql: command not found

Usually this means that psql isn’t in the PATH of the user.
Locate where psql is installed and export that path:

find / -name psql to determine psql location

For VCAC psql is located in /opt/vmware/vpostgres/current/bin/psql

export PATH=/opt/vmware/vpostgres/current/bin/:$PATH” resolved this issue.

Make sure to add this path to .bash_profile if you wish to make it permanent.

(c) Zbigniew ‘Ziggy’ Misiak

UIM/P 4.0.x is unable to install ESXi – PXE boot issue

When you’ll try provision ESXi using UIM version 4.0.x and Cisco UCS 2.1(3x) it might fail to install OS.

Installer hangs on s.v00 file and keeps rebooting.

There is an issue with the pxelinux third party software version utilized by UIM/P 4.0.0, UIM/P 4.0.0 Patch 1, and UIM/P 4.0.0 Patch 2, for OS installation.

To workaround this issue for the above releases, follow the instructions below:

  1. Go to PXE directory
    cd /opt/ionix-uim/bmp/deployment/PXE
  2. Save the original file to different file name
    mv pxelinux.0 pxelinux.0.bak
  3. Copy the older version 386.
    cp pxelinux-386.0 pxelinux.0
  4. Resume provisioning, no restart is needed.

If has been confirmed that this is not an issue in ECM UIM 4.1

(c) Zbigniew ‘Ziggy’ Misiak