Get all SharePoint Document Library Files and Folders at a ServerRelativeUrl in One REST Call
Recently, I was building a directory tree view of a Document Library for a client. Yes, you can say this shouldn’t be necessary. We can just tag the documents with metadata and we won’t need folders at all. Unfortunately, that’s not always the way people want to work.
To build this, I started by calling the _api/Web/Lists/getbytitle('Document Library Name')/items endpoint. I figured I’d just get all the documents in the library and sort out the display from the array. I got this working pretty well and in my test environment with a few hundred documents, it worked great.
Then – boom. There was a little requirement I didn’t know about: quite a few of the Document Libraries where we wanted to use this had more than 5000 documents. I stupidly hadn’t thought of that possibility. In my experience, it’s pretty unusual to see Document Libraries with that many documents, though it definitely happens.
Why does 5000 documents matter? Well, as of SharePoint 2010, we went from being able to request as many items as we wanted to an item limit of 5000. I’ll admit it’s not usually a great idea to request that many items from the client, but sometimes we need to.
I had two choices:
- Add some paging logic to the call. This would mean that if there were more than 5000 items, I’d simply make
Math.ceil(documentCount / 5000)calls to get them. In smaller Document Libraries, it would still be one call; as the number of documents went up, it would take more calls. - Be smart about it. Just request the objects (files and folders) in the root of the Document Library, and then on-demand, only request what I needed as the user expanded each folder.
The former would have been a little easier, but with larger libraries those calls could get pretty slow. The latter idea was really the “enterprise” way to do it. The problem was that the _api/Web/Lists/getbytitle('Document Library Name')/items endpoint didn’t really give me the right info to do it well.
So I turned to a different endpoint: _api/Web/GetFolderByServerRelativeUrl('folderRelativeUrl'). This is a newer endpoint and is designed for doing stuff like this. We can pass in the relative URL – maybe something like “/sites/SiteA/SubSiteA/LibraryName/TopFolder/SubFolderA/SubFolderB” – and get back just the files and folders for that relative path.
It takes two calls for this, though:
_api/Web/GetFolderByServerRelativeUrl('folderRelativeUrl')/Folders_api/Web/GetFolderByServerRelativeUrl('folderRelativeUrl')/Files
That would work, but it seemed a bit inefficient. Wouldn’t it be better to get the files and folders at the same time?
Off I went to Bingle. Luckily, I found a post on SharePoint StackExchange pretty quickly from jkr asking the same thing: Get all Files and Folders in one call. Vadim Gremyachev replied with the trick.
_api/Web/GetFolderByServerRelativeUrl('folderRelativeUrl')?$expand=Folders,Files
With this one call, we can get the info about the file and the folders together in one complex object.
As I said, this endpoint is perfect for building something like a directory tree.
There’s not a lot of good documentation for this endpoint (surprise!). You can find some examples of calls on the MSDN page Files and folders REST API reference, but no examples of the results. If you download the SharePoint 2013 REST Syntax (wall posters) you get some more clues.
The Files result provides results like those shown in Figure 2. As far as I can tell, there’s no way to control what fields you get back, as using $select has no effect.
The Folders result provides results like those shown in Figure 3. As far as I can tell, there’s no way to control what fields you get back here either, as using $select has no effect.
Note that in the Folders results, there are also Files and Folders objects, so the idea of recursion is there, though the objects are deferred. Because each folder has a ServerRelativeUrl value, you can dig as deep as you need to.
If you know you only need to go a few layers deep, you can also do things like:
_api/Web/GetFolderByServerRelativeUrl('folderRelativeUrl')?
$expand=Folders,Folders/Folders,Folders/Folders/Folders
or
_api/Web/GetFolderByServerRelativeUrl('folderRelativeUrl')?
$expand=Files,Folders/Files,Folders/Folders/Files
Both of these calls will get you three folders deep, which may be enough for some things you might want to do. I could also see using a call like these latter ones to get a bit ahead of your user to reduce the “chatter” on the line. That would make your array processing on the client side a little more complex, but could be worth it.
With some spiffy recursion in your framework of choice, you can build some very nice user interfaces with data like this. But that’s for another post…
Hi Mark,
Another way to do this is to use Web/Lists rest API and pass it a caml Query to filter out just the folder you want to see. You can see an example at https://github.com/russgove/SPSecurity (it even has the treeview).
sample code:
url = this.getHostApiUrl(“Web/Lists/GetByTitle(‘” + listTitle + “‘)/getitems?$expand=ContentType,Folder,Folder/ParentFolder,File,File/ParentFolder,RoleAssignments,RoleAssignments/RoleDefinitionBindings,RoleAssignments/Member,RoleAssignments/Member/Users,RoleAssignments/Member/Groups,RoleAssignments/Member/UserId”);
var caml = “” +
” ” +
“” +
” ” +
” ” +
” ” +
folderServerRelativeUrl +
” ” +
” ” +
” ” +
” ” +
// ” “+
// “” +
// “FolderUrls”+
//””+
” “;
var queryPayload = {
‘query’: {
‘__metadata’: { ‘type’: ‘SP.CamlQuery’ },
‘ViewXml’: caml
}
};
var folderLoaded = $q.defer();
var rd = document.getElementById(“__REQUESTDIGEST”);
$http.post(url, JSON.stringify(queryPayload),
{
headers:
{
“Accept”: “application/json; odata=verbose”,
“Content-Type”: “application/json; odata=verbose”,
“X-RequestDigest”: rd.value
}
})
@Russ:
Indeed. I’m trying my darnedest these days to use pure REST when I can.
M.
Hi Marc,
I am working on C# Web Application connected with SharePoint online via REST. I want to move a child folder with Metadata from one parent folder to other parent folder in same document library. Do you know how to achieve this? If possible kindly share the Rest call with __metadata payloads.
Regards,
Syed
Syed:
I’d usually turn to a third party tool like Sharegate for something like this rather than writing code.
M.
Thanks for the tutorial.
I’m trying to do the same but face a problem. When I make a query, the response that I get contains folders that are usually hidden in native SP. I don’t want to show those. Is there a way to filter them out?
@Dzhavat:
This method is good *because* it returns the files and folders. You can either use a different request or (probably more easily) just filter out the folders on the client. You’ll notice that the Files and Folders come back in totally separate nodes, so you can just ignore the Folders.
M.
Hi Marc,
I would love to know whether is it possible to get authentication token in Javascript to connect to Sharepoint ?
Thanks
Hi Marc, Thanks for your great article. This one and all the other ones. expand=Folders,Files work fine for us on SP 2016 in Azure. The only issue that we are facing is that it doesn’t support server side pagination. Consider we have lots of files and folder and we are using KendoUI to show it as a grid. For doing pagination we need to use Top and Skip in our REST query but it doesn’t support with GetFolderByServerRelativeUrl endpoint. Any idea?
@Ehsan:
Can’t you just retrieve all the files and folders and handle it client side? Unless your library is insanely large, that will be more efficient.
M.
I mean adding this $skiptoken=Paged=TRUE%26p_ID%3dXYZ won’t affect query results. so It doesn’t allow us to paginate result server side.
https://************.sharepoint.com/sites/SolutionCentral_qa/_api/web/lists/getbytitle('Submission%20Library‘)/Files?$select=Name&$Filter=Name%20eq%20{%27Publishdoc-prod.js%27}
throwing error : method is not supported.
Environment: In SharePoint Online.
Want to check if file is already existing in the library.
The same code was working fine for 2013 enviroment
https://****.sharepoint.com/sites/SolutionCentral_qa/_api/web/lists/getbytitle('Submission%20Library‘)/items?$select=Name&$Filter=Name%20eq%20{%27Publishdoc-prod.js%27}
Please advice Thanks,
@shivendra:
I don’t see anything obviously wrong, but odds are it’s a quote in a missing place or something. In what you’ve posted, you do have a “smart quote”, so that could be the issue.
M.
Hi Marc,
I am looking at REST API’s to use in Sharepoint 2013 which will allow for Upload a file to Document Library , Bulk Upload functionality , Check out n Check in on Documents and Download functionality .
Could you please assist me on the above topics ?
Regards,
N
@Nishal:
Those REST endpoints are fairly well documented by Microsoft. Use this article as a starting point.
M.
Hi Marc,
I am using REST api to upload document to Sharepoint using HTML . Despite giving all the permissions on Sharepoint end , i end up getting 401 UnAuthorised error while trying to upload the file . For more details please visit : https://stackoverflow.com/questions/44173820/401-error-while-uploading-file-to-sharepoint-2013-using-rest-api .
It would be great if you could assist me on this topic ? Please note i am using trial version of Sharepoint for POC . Not sure whether upload of Document is restricted or not ?
Regards,
N
Thank you very much for the informative article. It seems there is no way to sort the result if you use ‘$expand=Folders,Files’? The standard $orderby doesn’t do much.