SharePoint 2013 Ceres Shell – down the search rabbit hole

When preparing for the BIWUG Session – Everything you always wanted to know about SharePoint Search Relevance  I came across a number of blog posts about the Ceres shell. For those of you interested about the inner workings of SharePoint search, keep on reading.

The Ceres shell is a set of PowerShell cmdlets which allow you to control the internal workings of SharePoint Server 2013 search. It is a remnant of the FAST search engine which provided a set of very powerful tools to tweak the search engine inner working (but also to completely tear down you search :-)) and which is now integrated in SharePoint Server 2013.

As Christoffer Vig outlined in his blog post Making synonyms visible in SharePoint 2013 search results  modifying the settings in the Ceres engine can completely ruin your SharePoint install, so be careful and I’m not really sure if modifications are even supported.
In this post we will explore how the different content processing pipeline components work using the Ceres shell. The Ceres shell script is located underneath C:\Program Files\Microsoft Office Servers\15.0\Search\Scripts\ceresshell.ps1.  The script listed below allows you to connect to Ceres engine and list all the different flows.

Add-PSSnapin Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue

& "C:\Program Files\microsoft office servers\15.0\search\Scripts\ceresshell.ps1"

Connect-System -Uri (Get-SPEnterpriseSearchServiceApplication).SystemManagerLocations[0] -ServiceIdentity hpw\spfarm

Connect-Engine

Get-Flow

To look at the configuration of one specific flow you can specify the flow name as a  parameter.

Get-Flow Microsoft.MetadataExtractorSubFlow

One of the things I had been struggling with in SharePoint 2013 is the title field which is shown in the search results. By default SharePoint 2013 contains a mechanism which will override the title managed search property with some extracted text from PowerPoint and Word documents – this is similar to the EnableOptimisticTitleOverride functionality in SharePoint 2010. The Microsoft.MetadataExtractorSubFlow.dll contains some mechanism which determines that the some line of text in a header of a document is a more suitable title for the document and overwrites all of your other properties (The title filled in the Word document properties or the title in SharePoint, etc…). However, this does not always work as expected.

If you look at the flow details, you will see that extractedTitleField maps onto the Title managed search property and that the Author and LastModifiedTime fields use the same mechanism.

I guess that if you update this flow in a similar fashion as here Making synonyms visible in SharePoint 2013 search results you will be able to display the metadataextraction but I think there is a better way to do this after installing the SharePoint Server 2013 October 2013 Cumulative Update  - stay tuned for another update on this in a next  blog post.

If you have found great blog posts or other uses of the Ceres engine – please leave a comment.

 References:

• SharePoint 2013 Search internals: The Ceres shell
• Making synonyms visible in SharePoint 2013 search results – modifying the synonym weight using the SharePoint 2013 Ceres shell
• Extending the Ceres Engine with custom flows and operators

Changing social e-mail notification settings in SharePoint 2013 using the SharePoint server object model

I recently got the question whether it was possible to change the e-mail notification settings from the newsfeed functionality in SharePoint 2013 for all users – for those of you who don’t know this functionality check out Get email notifications about newsfeed items. Out of the box you will get a lot of e-mails for different actions on your newsfeed such as when someone has started following you, mentions, replies to conversations or community discusion posts, etc …

It actually is a UserProfile property called EmailOptin which controls this behavior but unfortunately there is not a lot of documentation about this property. The property contains an integer value that you need to set – the logic seems to be a bit strange but you need to set the value of the bit to 1 if you want the checkbox to be unchecked – the next example shows if you only want to

	Integer value to uncheck
Someone has started following me	2
Suggestions for people and keywords I might be interested in	4
Someone has mentioned me	8
Someone has replied to a conversation that I started	16
Somone replied to a conversation that I replied to	32
Someone replied to my community discussion post	64

 

This means that if you only want to uncheck the first option (Someone has started following me) you need to set it SPS-EmailOptin to 2. Listed below is a code sample for a console app which runs on the SharePoint server and which sets the SPS-EmailOptin property.

 

   1:  using System;

   2:  using System.Collections.Generic;

   3:  using System.Linq;

   4:  using System.Text;

   5:  using System.Threading.Tasks;

   6:  using Microsoft.SharePoint;

   7:  using Microsoft.Office.Server.UserProfiles;

   8:   

   9:  namespace UserProfilesServerOM

  10:  {

  11:      class Program

  12:      {

  13:          static string strUrl = "http://portal.hpw.local";

  14:          static string strAccount = @"hpw\joris";

  15:   

  16:          static void Main(string[] args)

  17:          {

  18:              

  19:              // Get SPSite and service context from string 

  20:              SPSite site = new SPSite(strUrl);

  21:              SPServiceContext serviceContext = SPServiceContext.GetContext(site);

  22:              

  23:              // Initialize user profile config manager object 

  24:              UserProfileManager upm = new UserProfileManager(serviceContext);

  25:              

  26:              

  27:              //To set prop values on user profile 

  28:              UserProfile u = upm.GetUserProfile(strAccount);

  29:              string sPropName = "SPS-EmailOptin";

  30:              //126 unchecks all options

  31:              u[sPropName].Value = 2;

  32:              u.Commit();

  33:   

  34:              Console.WriteLine("Hit any key to continue...");

  35:              Console.ReadLine();

  36:          }

  37:      }

  38:  }

Technorati Tags: SharePoint,SharePoint2013,User+profiles,Social

SharePoint 2013 search relevance session at BIWUG - December 2013

Presentation about search relevance at BIWUG (www.biwug.be) on the 18th of December - inspired and largely based on a session at SharePoint Conference 2012 in Las Vegas - check out the SPC145 Optimize Search Relevance in SharePoint 2013 video on Channel 9.

BIWUG session – SharePoint Apps and everything you always wanted to know about search relevance

On December 18th, BIWUG (www.biwug.be) is organizing another session about SharePoint apps as well as a deep dive session on SharePoint 2013 search – don’t forget to register for BIWUG1812.

Agenda:

• 18:00 - 18:30 ... Welcome and snacks
• 18:30 - 19:30 ... SharePoint Apps voor Developers  ( Speaker: Peter Plessers )

Samen met SharePoint 2013 werd ook het nieuwe App model geïntroduceerd die ons ontzettend veel mogelijkheden geeft voor het ontwikkelen van complexe business toepassingen voor zowel SharePoint Online als SharePoint On-Premise. In deze sessie gaan we aan de hand van een real-life toepassing in op een aantal onderwerpen die je kunnen helpen bij het ontwikkelen van SharePoint Apps.

De volgende onderwerpen komen aan bod:

• User & App authentication

• Integration between SharePoint & Windows Azure

• Using search and user profiles in your app

• App parts.

• 19:30 - 19:45 ... Break
• 19:45 - 20:45 ... Everything you always wanted to know about  ... ( Speaker: Joris Poelmans )

Search in SharePoint 2013 provides a comprehensive set of tools to manage search relevance. One of key enhancements is the introduction of query rules, that allow administrators to control relevance for a single query or sets of queries using a straightforward user interface. In this session, we will learn how ranking works under the hood, and give hands on demonstrations of how to use query rules, result sources, XRANK, federation to external search providers and the rank tuning tool to improve your relevance. We will see which tools are most appropriate for a situation and look at how you should approach search projects.

• 20:45 - …      ... SharePint!

 

Technorati Tags: SharePoint,biwug,community,search,apps

How to – Generate XML from CSV files

 

Great example on MSDN about how you can use Language-Integrated Query (LINQ) and LINQ to XML to generate an XML file from a comma-separated value (CSV) file. Download sample Visual Studio 2012 project

Technorati Tags: Visual+studio,c#,DotNet,XML,LINQ

How to turn the touch pad back on on HP portables

This week I accidently disabled the touch pad on my HP portable (EliteBook 8570p). Apparently this happened because I touched the upper left corner of the touch pad – you see an orange led light when touch pad is disabled. Just double tap the upper left corner again and it is activated again.

Technorati Tags: HP,Troubleshooting,touchpad

Fixing the form cannot be rendered error in SharePoint 2013 workflows

I recently encountered an error when trying out the standard SharePoint 2010 workflows in a newly installed SharePoint 2013 environment. When opening the workflow initiation form it gave me an error “The form cannot be rendered. This may be due to a misconfiguration of the Microsoft SharePoint Server State Service. For more information, contact your server administrator.”

Apparently my SharePoint 2013 did not have a State Service Application configured – to create a new State Service Application you are required to use PowerShell since the option is not available in the interface. Take a look at New-SPStateServiceApplication cmdlet documentation on Technet

   1: $serviceApp = New-SPStateServiceApplication -Name “State Service Application”

   2: New-SPStateServiceDatabase -Name “SharePoint_Service_State” -ServiceApplication $serviceApp

   3: New-SPStateServiceApplicationProxy -Name “State Service Application Proxy” -ServiceApplication $serviceApp -DefaultProxyGroup

Technorati Tags: SharePoint,administration,troubleshooting,microsoft,SP2013,SharePoint+2013

Understanding SharePoint 2010 Search ranking model and tuning SharePoint People Search ranking

In search, relevance is about how closely the search results that are returned match the user’s  intent– or as outlined in About intent, recall, relevance and precision of search solutions – relevant results will help you achieve the goals that made you perform the search in the first place. It is therefore quite important that you don’t need to navigate through dozens of pages of search results and that you get the most relevant results listed on the top of your search results.

SharePoint 2010 search has standard 9 different relevancy ranking models defined, from which two are used by default. To get an overview of the different ranking models which are registered you can use the Get-SPEnterpriseSearchRankingModel PowerShell cmdlet.

Get-SPEnterpriseSearchRankingModel -SearchApplication "Search Service Application"

After running this command you will get an overview of the different available ranking models.

The two default ranking models that are used are MainResultsDefaultRankingModel and MainPeopleModel. You can see this by exporting the Core search results web part and the People search results webpart. For example when exporting the People search results webpart you will notice the following property

So let’s take a closer look at the MainPeopleModel – at first I thought that I could simply export a ranking model to XML in a similar fashion as outlined in Rank models in 2013 – Main differences

$peoplerankingmodel = Get-SPEnterpriseSearchRankingModel -SearchApplication $ssa -Owner $owner -Identity d9bfb1a1-9036-4627-83b2-bb9983ac8a1 
$peoplerankingmodel.RankingModelXML > peoplerankingmodel.xml

But unfortunately this XML returned seemed to be empty on my SharePoint 2010 environment – which seems to be behavior by design as explained in this blog post Accessing rank models in SharePoint 2010. But luckily you can also open the MSSRankingModels table in the Search Service Application database and take a look at the XML which is stored in the database.

Now let’s take a closer look at the XML which I copied out. As you see in the XML SharePoint Server 2010 also implements the BM25 ranking model which gives weights to certain fields (managed search properties) for dynamic ranking (This is a very simplistic explanation – for detailed explanation see Microsoft Cambridge at TREC-14: Enterprise track and Static Score Bucketing in Inverted Indexes )

You see a number of managed search properties which are being used – these actually map onto the following crawled properties.

Managed property	Mapped crawled property
RankingweightName	People:CombinedName,People:UserName,People:WorkEmail,People:AccountName,People:SPS-SipAddress
PreferredName	People:PreferredName
JobTitle	People:SPS-JobTitle,People:Title,ows_JobTitle,ows_Job_x0020_Title
Responsiblities	People:SPS-Responsibility (Ask me about)
RankingWeightLow	People:SPS-Skills, People:SPS-Interests
ContentsHidden	People:SPS-Location,People:Office,People:SPS-PastProjects,People:SPS-School, etc…
Memberships	People:Quicklinks
RankingWeightHigh	People:OrganizationNames,People:Department
Pronunciations	People:SPS-FirstName,People:SPS-LastName,People:SPS-PreferredName,People:SPS-PhoneticFirstName, etc …

So you will probably notice that the People search will not take your own custom created user profile properties into account when doing a search for people. You will also notice that the user profile properties skills and interests are assigned the same weight from a relevance perspective since they both map onto the RankingWeightLow property.

It is however also possible to create your own ranking model based on the Ranking Model Schema definition (MSDN). But Microsoft will only allow you to customize certain parts of the search engine ranking algorithm (For SharePoint 2013 there is a lot more in depth documentation of the ranking models – check out Customizing ranking models to improve relevance in SharePoint 2013 ).

In the next example XML schema I added my own custom managed property Certifications and kept all of the other properties and weights for the different querydependent features.

To get at the internal id of the property (called pid in the example above) – you will need to use the Get-SPEnterpriseSearchMetadataManagedProperty PowerShell cmdlet. Besides the weight you can also specify lengthnormalization – a good explanation of this is found in Evaluation and customizing search relevance in SharePoint 2007 :

Modifying the length normalization setting applies only to properties that contain text. For example, consider a scenario where relevance is calculated for two content items containing the query term within the body of the content item, within a book, and within a document containing only a short paragraph. The book is likely to contain more instances of the query term, so it might receive a higher rank value, even if the shorter document is just as relevant to the user's search. The length normalization setting addresses this issue, providing consistency in ranking calculations for text properties, regardless of the amount of text within that property.

Next we will need to import our XML with the custom ranking model – you do this using the New-SPEnterpriseSearchRankingModel cmdlet.

The easiest way to test your new custom ranking model is by using a querystring parameter rm – like this http://jopx/searchcenter/peopleresults.aspx?k=yammer&rm=[ID of your ranking model]. In the screenshot below the People search will also take into account my own custom profile property - Certifications allowing me to search on them.

To use this new model in production, you can export the Core Results webpart, modify the RankingModel parameter and reimport the web part.

References:

• Improve SharePoint Search Relevance – interesting sample code for building a console app which uses a custom ranking algorithm
• Microsoft Cambridge at TREC-14: Enterprise track
• Customizing ranking models to improve relevance in SharePoint 2013
• Ranking Model Schema definition (MSDN)
• Rank models in 2013 – Main differences
• Tuning SharePoint Search Relevance with Ranking Models

Seminarie - Anders vergaderen, communiceren en samenwerken

Het nieuwe werken brengt twee belangrijke risico’s met zich mee, wanneer het té nauw geïnterpreteerd wordt. Minder innovatie en een lagere betrokkenheid. Twee valkuilen die een enorme impact hebben op het slagen of falen van de implementatie van een nieuwe werkplek. Drie succesfactoren zijn cruciaal: de juiste tools om mee te werken, een gepaste kantooromgeving en een gedrags- en cultuurwijziging, begeleid vanuit HR.

Tijdens dit Anders Vergaderen Event dompelen we u - wetenschappelijk onderbouwd door Professor P. Vink (TU Delft) - onder in de wereld van het nieuwe werken en zijn bottlenecks. Een combinatie van praktijkvoorbeelden, business sessies en het aan-den-lijve-ondervinden maakt dat het een boeiende voormiddag wordt, waarna u met een hoofd vol inzichten de herfstvakantie kunt  instappen.

Agenda

• Het Nieuwe Werken vraagt om anders bijeenkomen  - Spreker: Peter Vink is hoogleraar aan de faculteit Industrieel Ontwerpen van de TU-Delft en houdt zich bezig met “environmental design”. Hij heeft meer dan 250 artikelen en boeken geschreven over het effect van de omgeving op comfort en prestatie.
• Change management om resultaat te boeken Werkelijk resultaat boeken met het Nieuwe Werken lukt enkel als de invoering aangepakt wordt als een integraal verandertraject. Hierin is Acerta deskundig. Wij laten u graag kennismaken met onze changemanagement methodologie en toolbox. – Spreker: Veerle Theys, Director Change Management, Acerta
• The connected company 66% van de CIOs zien efficiënte samenwerking als essentieel voor waarde creatie. Bovendien zijn het merendeel van de taken van kenniswerkers dermate complex geworden dat samenwerking tussen verschillende disciplines binnen een bedrijf en over bedrijven heen noodzakelijk is. RealDolmen ligt in deze sessie toe hoe technologie samenwerking kan ondersteunen en wat de valkuilen en best practices zijn bij de implementatie van samenwerkingsplatformen – Spreker Joris Poelmans Principal Consultant, RealDolmen
• Het werkelijke connectieverhaal Hoe zorg je ervoor dat mensen elkaar ‘vinden’ op het werk? Buroncept begeleidde reeds Decathlon , Toyota, Taminco, Michael Page, Sony Music en Petercam naar hun ideale connectieplek. Spreker: Ines Van Obbergen, Project Consultant, Buroconcept

Datum
Vrijdag 25 oktober van 9.30 tot 14.00 uur.

Deelname is gratis. Voor annulaties na 18 oktober bedraagt de annulatiekost wel 50€.
Schrijf u hier in voor dit event.

 

Technorati Tags: HNW,Het+Nieuwe+Werken,HPW,RealDolmen

Co-authoring in SharePoint 2013 – things you should now

The co-authoring feature in SharePoint allows multiple users to work on a specific document simultaneously. This is enabled by a client-server technology which was first introduced in Office 2010 and SharePoint 2010 called Cobalt which allowed parts of an Office documents to be synchronized back to SharePoint. This technology was initially created to optimize network traffic from the clients to Office since Office would only send updates or the compressed differentiation (or delta) of the file back when saving a file.  

   Co-authoring is not supported on all of the Office documents in all of the different clients and it also seems that there is not a lot of change between 2010 and 2013. Things which might not seem so obvious:

• There also does not seem any difference in co-authoring experience when you compare SharePoint 2013 with Office Web Apps and SharePoint 2010 with Office Web Apps.
• For both SharePoint 2010 and SharePoint 2013 – no co-authoring available from Office clients (not in Excel 2010 nor in Excel 2013)
• Co-authoring is available using the Excel Web Apps – both in SharePoint 2010 and SharePoint 2013
• Co-authoring requires that the check-in/check out functionality is not enabled on SharePoint document libraries
• Co-authoring will work in most browsers (Internet Explorer, Firefox, Safari and Chrome) and on most platforms.
• Theoretically there is no limit on the number of users who can co-author a document.
• Office 2010 users have the same set of co-authoring features when they open documents from SharePoint 2013 or SharePoint Online as they do when they open documents from a SharePoint 2010 document library. 

References:

• Overview of co-authoring in SharePoint 2013
• Document collaboration and co-authoring  - more end user focused documentation

Nominated as SharePoint MVP for 2013 – 9 times in a row

I just got the news that I have been renewed as SharePoint Server MVP. I became a SharePoint MVP in 2005 – so this is my 9th year in a row as SharePoint MVP (For those who don’t know what MVPs are check out http://mvp.microsoft.com/en-us/overview.aspx) . I like to say that I’m proud to be a part of such a great community.

As SharePoint developer,consultant, architect or system engineer it currently is an exciting time with the introduction of the SharePoint app model, the cloud first approach of Microsoft around SharePoint and the integration of great technologies such as FAST and Yammer. So reach out to your local community groups to learn from your peers and discuss the challenges and best practices around SharePoint and Yammer – be it on-premise or in the cloud.

If you are working with SharePoint and living in Belgium – definitely register yourself on the BIWUG site - http://www.biwug.be/ to stay up to date for the latest events.

 

Technorati Tags: SharePoint,MVP,Community,BIWUG,Microsoft

SharePoint 2013 Search survey

Search is about more than just showing relevant information based on a keyword. It is also about visualizing and sharing knowledge and insights within an organization. The combination of Social and Search enables your employees to respond faster and find the experts in your organization.
Microsoft SharePoint 2013's new unified search platform (in which FAST Search has been integrated) ensures that the information can easily be found. The search engine uses new algorithms to continuously increase the relevance of the search results and makes it easier to integrate business applications. What arouses our curiosity however, is how frequently SharePoint's search function is actually used within your company. Hence the reason for this survey.  This way, we hope to gain a better insight into the current use of SharePoint Search, as well as the industry's present needs regarding this function.   SharePoint 2013 Search Survey  Are you curious about these results too? Participate in this study by filling out the survey and the results will be sent to you within a few weeks!   Completing the survey should only take about 5 minutes and, in addition, gives you a chance to win a 50€ voucher at Media Markt (Participants from Belgium only) Your contribution would be greatly appreciated.
Thanks in advance!

Technorati Tags: SharePoint+2013,Search,SP2013,Microsoft,survey

Site policies in SharePoint 2013

SharePoint sites rarely seem to be deleted even when they are not used anymore. This makes that the number of sites (and also the required space) can increase very quickly in a SharePoint environment. Site policies in SharePoint 2013 are designed to keep the number of sites in your SharePoint environment under control. It allows you to define through a policy when a site can be closed (and afterwards possibly deleted).

 You can define site policies for a site collection in the root site; these site policies are then available to all sites in the site collection. Go to Site Collection Administration and underneath the Site Settings section you will find the Site Policies page.  Although the documentation on MSDN - Overview of site policies in SharePoint 2013 -  states that there are 4 site policy options – the fourth one – run a workflow - is not available any more as this option has been removed in the final release – this leaves us with 3 options:

1. Do not close or delete site automatically. This policy option specifies that the site owner must manually delete the site.
2. Delete sites automatically. This policy option specifies that a site owner can close the site manually, but the site will be deleted automatically based on the deletion event settings specified.
3. Close and delete sites automatically. This policy option has the same choices as the preceding setting, for deleting sites automatically, but it also enables you to specify how long after a site’s creation date the site will be closed.

Since site policies are defined at site collection, SharePoint provides a mechanism to push out site policies across site collections using the content type hub. For more details on this check out this excellent blog post  – Site policy in SharePoint 2013

Technorati Tags: sharepoint,sharepoint+2013,sp2013,microsoft,governance,site+policies,ECM,ITPro,administration

SharePoint Server 2013 licensing and external users

 

Microsoft has introduced an important change in their licensing model for SharePoint 2013 by removing the need to acquire client access licenses (CAL) for external users in extranet and internet sites scenario. Yes, that’s correct no CALs are required for external users in SharePoint 2013. For some more info about licensing check out the links below:

• SharePoint 2013 licensing overview page
• SharePoint feature availability across on-premises solutions (Technet) – which compares Standard edition vs Enterprise edition is part of the SharePoint Online Service Description, also provides a clue about what Microsoft is currently focusing on.
• Licensing Brief – Licensing Microsoft SharePoint Server 2013 – explains common usage scenarios such as intranet, extranet and internet

Technorati Tags: SharePoint,SharePoint+2013,SP2013,Microsoft,licensing

Create a mapping to the SharePoint 2013 master page gallery on Windows Server 2012

SharePoint Server 2013 allows you to use your design tool of choice (Microsoft Expression, NotePad++, Dreamweaver, etc…) to create a new look and feel for your SharePoint 2013 environment (Check out this excellent post from Steve Peschka – Using Dreamweaver and Design Manager with SharePoint 2013 )

The look and feel of a SharePoint site is defined by a number of different artifacts and most of these are stored in the SharePoint master page gallery. One of the changes that Microsoft has introduced  to allow for this choice of design tools is to provide WebDAV support for the SharePoint Master Page Gallery (See Master pages, the Master Page Gallery and page layouts in SharePoint 2013 on MSDN for more background info)

If you are a SharePoint developer – you are probably working on the SharePoint Server itself (either Windows 2008 R2 or Windows Server 2012) and to make this integration available you will first need to activate the Desktop Experience feature on the server. On Windows Server 2012 this feature is visible underneath User Interface and infrastructure

After installing the Desktop Experience (this will require a reboot) you can simply open the SharePoint 2013 master page gallery in Windows Explorer.

Technorati Tags: SharePoint,sp2013,sharepoint+2013,development,windows+server+2012,microsoft,visual+studio,design+manager,customization

Managing SharePoint Online with Powershell

With the release of SharePoint 2013 as part of Office 365, Microsoft has also introduced the ability to manipulate SharePoint site collections in the cloud using PowerShell. Before you can start with the SharePoint Online cmdlets you have to install them separately from this link - http://www.microsoft.com/en-us/download/details.aspx?id=35588 . This downloads allows you to run SharePoint Online PowerShell from any host machine. It only requires PowerShell v3 or v4. SharePoint does not need to be installed locally. To use  the SharePoint Online cmdlets, you must must be a global administrator in Office 365.
Once installed you can open the SharePoint Online Management Shell,  it is implemented as  a PowerShell module which also allows you to use it in another PowerShell editor by manually loading the required module Import-Module Microsoft.Online.SharePoint.PowerShell.
To establish a connection to your SharePoint Online tenant you will need to use the Connect-SPOService cmdlet (Remember that you must use the tenant admin site url as a parameter). This will fill up a static internal variable which will be used afterwards. You should clear this internal variable using the Disconnect-SPOService cmdlet.

    Connect-SPOService –Url https://yoursite-admin.sharepoint.com –credential ray.charles@yoursite.onmicrosoft.com

Once you are connected you can manipulate your SharePoint Online environment. There are only 30 cmdlets available in for SharePoint Online – to get a full list type the following command.

Get-Command –Module Microsoft.Online.SharePoint.PowerShell

All cmdlets start with SPO. Most cmdlets are related to site collection management, user and app management. All returned object are simple data objects and most of them are limited in functionality. 

Cmdlet name	Description
Add-SPOUser	Adds an existing Office 365 user or an Office 365 security group to a SharePoint group.
Connect-SPOService	Connects a SharePoint Online global administrator to a SharePoint Online connection (the SharePoint Online Administration Center). This cmdlet must be run before any other SharePoint Online cmdlets can run.
Disconnect-SPOService	Disconnects from a SharePoint Online service.
Get-SPOAppErrors	Returns application errors.
Get-SPOAppInfo	Returns all installed applications.
Get-SPODeletedSite	Returns all deleted site collections that are in the Recycle Bin.
Get-SPOSite	Returns one or more site collections.
Get-SPOSiteGroup	Returns all the groups on the specified site collection.
Get-SPOTenantLogEntry	Retrieves SharePoint Online company logs.
Get-SPOTenantLogLastAvailableTimeInUtc	Returns the time when the SharePoint Online organization logs are collected.
Get-SPOUser	Returns the SharePoint Online user or security group accounts that match given search criteria.
Get-SPOWebTemplate	Shows all site templates that match the given identity.
New-SPOSite	Creates a new SharePoint Online site collection for the current company.
New-SPOSiteGroup	Creates a new group in a SharePoint Online site collection.
Remove-SPODeletedSite	Removes a SharePoint Online deleted site collection from the Recycle Bin.
Remove-SPOSite	Sends a SharePoint Online site collection to the SharePoint Online Recycle Bin.
Remove-SPOSiteGroup	Removes a SharePoint Online group from a site collection.
Remove-SPOUser	Removes a user or a security group from a site collection or a group.
Repair-SPOSite	Checks and repairs the specified site collection and its contents.
Request-SPOUpgradeEvaluationSite	Requests to create a copy of an existing site collection for the purposes of validating the effects of upgrade without affecting the original site.
Restore-SPODeletedSite	Restores a SharePoint Online deleted site collection from the Recycle Bin.
Set-SPOSite	Sets or updates the values of one or more properties for a site collection.
Set-SPOSiteGroup	Updates the SharePoint Online owner and permission level on a group inside a site collection.
Set-SPOTenant	Sets properties on the SharePoint Online organization.
Set-SPOUser	Configures properties on an existing user.
Test-SPOSite	Tests a SharePoint Online site collection.
Upgrade-SPOSite	Starts the upgrade process on a site collection.

There are no cmdlets available for managing SharePoint objects at a lower scope than the site collection. The following examples show you some examples of the more common cmdlets. To see a list of site collections associated with a subscription or to see the details for a specific site collection use the Get-SPOSite cmdlet. Use the following command to retrieve the details about a specific SharePoint Online site collection. You need to specify the –Detailed option to retrieve information about CompatibilityLevel,ResourceUsageCurrent,ResourceUsageAverage, StorageUsageCurrent,Webcount and the Title.

Get-SPOSite https://yoursite.sharepoint.com –Detailed | select *

A slight variation on the script above allows you to get usage data about all of your different SharePoint Online site collections. The use of –limit all allows you to get all of the SP Online site collection, standard it will only return 200.

Get-SPOSite –limit all –detailed | Export-CSV –path MyReport.csv

There is however a workaround for manipulating objects in SharePoint Online at a lower level using the SharePoint Server 2013 Client Components SDK which enables remote development against SharePoint Server 2013. The Client Side Object Model (CSOM) is an API which allows you to do remote development against SharePoint in a fashion quite similar to the way that you would program on the server (but it is a subset and therefore does not expose all of the same classes).  There are 3 implementations of this API  - one for .NET, one for Silverlight and one for Javascript and they are meant to replace the SharePoint web services. The .Net managed model installs a number of dlls (all using the Microsoft.SharePoint.Client.* namespace) underneath

C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI.
The next sample code shows how you can use these assemblies from Powershell which allow you to do a lot more with PowerShell and SharePoint Online.

$loc = "C:\Powershell" # Location of DLL's

$siteUrl = "https://yoursite.sharepoint.com"

$loginname = "ray.charles@yoursite.onmicrosoft.com"

Set-Location $loc

Add-Type -Path (Resolve-Path "Microsoft.SharePoint.Client.dll")

Add-Type -Path (Resolve-Path "Microsoft.SharePoint.Client.Runtime.dll")

Write-Host "Please enter password for $($siteUrl):"

$pwd = Read-Host -AsSecureString

$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteUrl)

# Authenticate against SharePoint Online

$ctx.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($loginname, $pwd)

$web = $ctx.Web 

$ctx.Load($web) 

$ctx.ExecuteQuery() 

Write-Host " Current web title is '

$($web.Title)', $($web.Url)"

 

In the next blogpost I will show some additional examples of how you can use PowerShell in combination with the SharePoint CSOM.

 

Technorati Tags: SharePoint,SharePoint+2013,sp2013,Powershell,Office365,SharePoint+Online,SPO,administration,Microsoft

Multilingual User Interface and Language Packs for SharePoint Server 2013

The Language Packs for SharePoint Server 2013 are now also publicly available for download (and not only from MSDN as in the previous months) – go to Language Packs for SharePoint Server 2013 [English] and select the appropriate language in dropdown. Also take a look at Install or uninstall language packs for SharePoint Server 2013 before performing the installation. Remember that you are no longer required to first do an installation of the SharePoint Foundation language packs.

From an end-user perspective there is a big change with regards to the multilingual interface. In SharePoint 2010 an end user could quite easily switch his language using the language picker in the top right of the page (See Multilingual User Interface in SharePoint 2010 for more details).
Unfortunately this is not the case anymore for SharePoint 2013. By default a SharePoint site for which alternate languages are defined will be shown in the language which is defined in the browsers language settings. A user can however change this by editing  his SharePoint user profile – Select About Me (in the top right corner), Edit Profile and next expand the tab menu by clicking the 3 dots and select Language and Region. In the next screen a user is able to select his preferred display language.

Once a user has selected a specific language – the following elements in the user interface will be translated (as shown with the French display language below)

• The standard SharePoint menus e.g. the ribbon are translated
• Navigation menu’s also support multilingual scenario’s with the MUI – if you switch your language and translate specific menu nodes in your navigation – these changes are language specific.
• The headings for list and site columns
• The managed metadata field type also supports multilingual scenario’s.

In my opinion this is making things way to complicated for end users and we will probably need some other workarounds for automatically setting the display language of a user since the majority will not find these menu options.

Technorati Tags: sharepoint,sp2013,MUI,multilingual,sps2013,sharepoint+2013,microsoft

Big Data – reading and video material

Mental note to myself to take a look at this in the coming weeks :-) …

• Understand your customers with Hadoop – tutorial – analyzing sentiment data
• Hortonworks Sandbox – a full HDP environment on a virtual machine with a dozen interactive hadoop tutorials to get you started
• Where does Hadoop fit in a modern data architecture
• The next big thing in Big Data: People Analytics
• Analyzing Big Data with Twitter (UC Berkeley course lectures)
• Big Data analytics e-book (IBM)
• Big Data Analytics – Facts & feelings
• Big Data can bring patients to water but it can’t make them think – how Aetna uses Big Data to predict cardiovascular diseases
• The not so distant future of  Big Data
• Meeting the Challenge of Big Data (Oracle)
• Measuring return on Knowledge in a Big Data world
• 5 ways Big Data is going to blow your mind and change your world
• Microsoft’s vision of our future is big screens and big data
• SQL Server Blog – Busting Big Data Adoption Myths – Part  1, Part 2 and Part 3
• 3 presentations on R: Mining, Web Development and Data Visualization

Technorati Tags: Bigdata,mining,analytics

Using Information Rights Management in SharePoint Online

With the new Office 365, Microsoft has introduced a new functionality to secure documents using Information Rights Management (IRM) services. It is a persistent file-level technology that stops sensitive information from being printed, forwarded, downloaded or copied by unauthorized users. In Office 365 IRM is available as part of the Enterprise E3 and E4 plan or the Academic Plan 3 and Plan 4. The functionality is similar to Windows Right Management Server (RMS)  – see http://technet.microsoft.com/en-us/library/cc771234(v=ws.10).aspx for more details about RMS in an on premise deployment. When documents are downloaded from an IRM protected SharePoint document library, the supported file types carry the restrictions (in the form of an IRM license) along with the document as part of its content. Supported file types also include PDF next  to the standard Office file types in SharePoint 2013. The IRM protected files are encrypted and rights are restricted to the authenticated user who downloaded the document. IRM protection of PDF files is an extension to the existing ISO 32000 standard (See Microsoft IRM protection for PDF specification for more details) and needs to be implemented by the PDF readers – for the moment it is supported by FoxIT PDF line of products. To be able to use Information Rights Management (IRM) or Windows Azure Rights Management Services (the commercial name for IRM) in SharePoint Online there are 3 major steps required:

1. First enable Windows Azure RMS on Office 365 level – see Office 365 – Use Right Management Services. RMS is a shared service which can be used by Exchange Online and SharePoint Online and needs to be enabled at tenant level. It is not default enabled.
2. Next Set up Information Rights Management (IRM) in SharePoint admin center (Office 365). If you get an error here saying “Error: RMS Online is not enabled for this tenant, please contact Office 365 to enable.” – you probably forgot step 1.
3. Finally configure Rights Management on specific Document Libraries in SharePoint Online.

The IRM permissions map to SharePoint permissions on the document library as outlined in the following table

SharePoint Permissions	IRM Permissions
Manage Permissions Manage Web	Full control of the documents. This allows the user to read, edit, copy, save and modify permission of the document
Edit List Items Manage List Add and customize pages	Edit, copy and save permissions. The user can print the document only if the document library IRM settings are configured to allow document printing
View List Item	Read permissions. The user can read the document but not copy or edit its content. The user can  print the document only if the document library IRM settings are configured to allow document printing
All other permissions	Not applicable, no corresponding IRM permissions

In the IRM permissions can define additional options such as specifying whether documents that do not support IRM protection can be uploaded to the library and whether or not the document can be viewed in the browser. You can also configure additional document access rights which includes rights to print, run scripts to enable screen reader or enable writing to a copy of the downloaded document. The group protection and credential intervals determines the caching policy of the license that applications will use to open the documents. You can also enable sharing of the downloaded documents with users in a specified group.

Before you start implementing IRM in SharePoint Online you should carefully plan for it and define specific usage scenario’s for it since it is not meant to be activated on all information in discriminatory – ask yourself some of the following questions:

• Which business areas use sensitive information which is frequently exchanged?
• What needs to be protected (Office documents, e-mails, CAD design drawings, etc ..)
• How will security policies by applied and how will enforce and control that procedures are being followed.
• …

The introduction of IRM should be part of an overall information architecture and should be viewed upon at as part of a risk management strategy – ask yourself what would be the impact and consequence of information ending up in the wrong hands…
References:

• What’s new with Information Rights Management in SharePoint and SharePoint Online
• Apply Information Rights Management to a list or library (SharePoint 2013 and SharePoint Online)

Host-named site collections in SharePoint 2013

 

One of the recommendations in SharePoint 2013 is the usage of Host Named Site Collections (HNSCs) over using multiple web applications when you want to use vanity urls for specific SharePoint sites. Host-named site collections each have a unique DNS host name and therefore its own top-level URL.  SharePoint 2013 allows you to add multiple URLs by using the Set-SPSiteURL Powershell command. An interesting quote around this is found in a new article Host-named site collection architecture and deployment (SharePoint 2013) :

Because the Office 365 environment uses host-named site collections, new features are optimized for these site collections and they are expected to be more reliable. Learn how to plan for and implement host-named site collections, design URLs, and manage URLs.

The same information is found in SharePoint 2013 design samples – corporate portals and extranet sites :

The recommended configuration for deploying sites is using host-named site collections with all sites located within a single web application. This configuration is recommended to deploy sites because it is the same architecture that the Office 365 environment uses. Consequently this is the most heavily tested configuration. New features, including the App model and Request Management, are optimized for this configuration, and it is the most reliable configuration going forward.

Some of the reasons why host-named site collections are also considered to be the better solution because:

Create less overhead on web servers, because no additional IIS web site is required

Provide more flexibility because you provide alternative access to individual site collections instead of entire web applications. There is a limit of 20 web applications per farm (with 5 predefined zones), but you can create thousands of site collections (each with their own vanity url)

Reduced resource consumption since there are less web applications required (ant thus also less application pools)

Mitigates cross-site scripting risks

Other references:

• Your friendly guide to host-named site collections
• What every SharePoint Admin needs to know about Host Named Site Collections
• Host-named site collections (HNSC) for SharePoint  architects

Technorati Tags: SharePoint,administration,sp2013,microsoft,ITPro,architecture

SharePoint Server 2013, Active Directory Groups and the access denied problem caused by token lifetime.

A while ago we experienced some issues when using Active Directory groups and SharePoint Server 2013. We added users to Active Directory groups which were added to SharePoint groups, but the users who got access using this method, still got access denied. After some investigation we found the problem.

In SharePoint 2013 all authentication is being handled through claims which are managed by the Security Token Service (STS). The STS is responsible for issuing security tokens to users. A user which has logged on to SharePoint will be authenticated and granted a token with his group membership. These tokens have a certain lifetime and will not be updated until they expire. The property which controls this is the WindowsTokenLifetime property – according to the documentation is it normally set to 1380 minutes or 23 hours -  see Set-SPSecurityTokenServiceConfig for more details. (On one of my SharePoint 2013 it seemed to be 10 hours).

Therefore users who had already logged on to the SharePoint Server did not get the required tokens for their new AD group membership until the token expired. You can however change the token expiration in the following way:

$sts = Get-SPSecurityTokenServiceConfig
$sts.WindowsTokenLifetime = (New-TimeSpan -Minutes 15)
$sts.LogonTokenCacheExpirationWindow = (New-TimeSpan -Minutes 15)
$sts.Update()

Changing the WindowsTokenLifeTime will probably have a performance impact since there will be a lot more requests going from the STS to your Active Directory domain controllers. I haven’t found clear guidance or best practices about this, so feel free to leave a comment.

Technorati Tags: SharePoint2013,sp2013,SharePoint,administration,Active+Directory,claims+authentication

Community Day 2013: Building search-driven Windows 8 and Windows Phone 8 apps for SharePoint Server 2013

Building search-driven Windows 8 and Windows Phone 8 apps for SharePoint Server 2013 from Joris Poelmans

Technorati Tags: SharePoint,SP2013,WP8,Windows8,Search