Ah OK, thanks. So I should use U() for literal JSON keys then, and json::value(conversions::to_string_t(somestring)) for values? The last one is a bit of a mouthful...
↧
New Post: Should I use L or U() when handling strings with JSON?
↧
New Post: extract_json performance
Hi Ganesha,
The default chunksize is already 64K. Also I assume you are running with the release build, correct?
In your prior examples you don't have everything quite right, if you want to use the set_response_stream try something like the following:
Steve
The default chunksize is already 64K. Also I assume you are running with the release build, correct?
In your prior examples you don't have everything quite right, if you want to use the set_response_stream try something like the following:
web::http::client::http_client_config config;
config.set_chunksize(1024 * 1024); // 1 MB chunks
web::http::client::http_client client(U("http://www.bing.com"), config);
// Write the response directly to a std::string backed buffer.
// Reserve all the memory needed upfront in one heap allocation.
concurrency::streams::container_buffer<std::string> buffer;
buffer.collection().reserve(4508876);
web::http::http_request request(web::http::methods::GET);
request.set_response_stream(concurrency::streams::ostream(buffer));
// Send request, please note this is blocking synchronously here.
// The task returned from http_response::content_ready() indicates
// when the entire response body has arrived.
web::http::http_response response = client.request(request).get();
response.content_ready().wait();
// Move the string out of the buffer to avoid a copy.
std::string responseBody = std::move(buffer.collection());
Steve
↧
↧
New Post: Should I use L or U() when handling strings with JSON?
Yes in both cases if you want to write entirely cross platform code with the library. In some of the APIs where both UTF-8 and UTF-16 is support, like with http_request::set_body.
Steve
Steve
↧
Commented Issue: uri_exception is thrown when uri contains square brackets [382]
`uri_builder` in `windows_request_context::read_headers_io_completion` throws web::uri_exception when the uri contains square brackets.
The encode_uri method handles square brackets as general delimiters and therefor will not encode them.
The encoded uri is passed to the uri_builder that throws the web::uri_exception.
The following code snippet reproduces the exception:
```cpp
auto encodedUri = uri::encode_uri(L"http://localhost/service[1]?param[]=123¶m[]=456");
// assert(encodedUri == L"http://localhost/service%5B1%5D?param%5B%5D=123¶m%5B%5D=456");
uri_builder builder(encodedUri);
```
In compare, the JavaScript `encodeURI` method encodes the square brackets as this exampel shows:
```javascript
console.log(encodeURI("http://localhost/service[1]?param[]=123¶m[]=456"));
// -> http://localhost/service%5B1%5D?param%5B%5D=123¶m%5B%5D=456
```
Is there a reason to keep the brackets unencoded and why is the behavior different to javascript?
Comments: Hi Markus, If the URI coming across wire has the square brackets percent encoded then the http_listener will leave the string as is and construct a web::uri object out of it. In that case the uri constructor won't throw an exception verifying. Of course if the URI coming across the wire has square brackets not percent encoded then the listener will respond with a bad request HTTP status. Perhaps I'm missing or overlooking something here? Thanks, Steve
The encode_uri method handles square brackets as general delimiters and therefor will not encode them.
The encoded uri is passed to the uri_builder that throws the web::uri_exception.
The following code snippet reproduces the exception:
```cpp
auto encodedUri = uri::encode_uri(L"http://localhost/service[1]?param[]=123¶m[]=456");
// assert(encodedUri == L"http://localhost/service%5B1%5D?param%5B%5D=123¶m%5B%5D=456");
uri_builder builder(encodedUri);
```
In compare, the JavaScript `encodeURI` method encodes the square brackets as this exampel shows:
```javascript
console.log(encodeURI("http://localhost/service[1]?param[]=123¶m[]=456"));
// -> http://localhost/service%5B1%5D?param%5B%5D=123¶m%5B%5D=456
```
Is there a reason to keep the brackets unencoded and why is the behavior different to javascript?
Comments: Hi Markus, If the URI coming across wire has the square brackets percent encoded then the http_listener will leave the string as is and construct a web::uri object out of it. In that case the uri constructor won't throw an exception verifying. Of course if the URI coming across the wire has square brackets not percent encoded then the listener will respond with a bad request HTTP status. Perhaps I'm missing or overlooking something here? Thanks, Steve
↧
New Post: extract_json performance
Hi Steve,
Even with this code, the performance has remained same (~9 sec).
I have Windows Server 2012 where http_listener is running. http_client is running on Windows 7. (And there is no virus scanner running on these test systems.)
Any possibility that http_listener of the REST SDK has some issue?
Ganesha
Even with this code, the performance has remained same (~9 sec).
I have Windows Server 2012 where http_listener is running. http_client is running on Windows 7. (And there is no virus scanner running on these test systems.)
Any possibility that http_listener of the REST SDK has some issue?
Ganesha
↧
↧
New Post: extract_json performance
Hi Ganesha,
Potentially, the focus of this library is really on the client side connecting portions. We have done very little performance work on the http_listener and don't have any current plans to. What kind of performance do you get if you remove the http_listener and just use your existing ATL server with our http_client?
Steve
Potentially, the focus of this library is really on the client side connecting portions. We have done very little performance work on the http_listener and don't have any current plans to. What kind of performance do you get if you remove the http_listener and just use your existing ATL server with our http_client?
Steve
↧
New Post: extract_json performance
Hi Steve,
ATL Server is an old technology from Microsoft which uses SOAP for communication (supported until VS2005). This is not compatible with http_client unfortunately.
Have you done any performance analysis of http_client with WCF as the http listener?
Is there any possibility that the focus on http_listener in REST SDK will never be picked up in future and will it get dropped out from the future plan?
Ganesha
ATL Server is an old technology from Microsoft which uses SOAP for communication (supported until VS2005). This is not compatible with http_client unfortunately.
Have you done any performance analysis of http_client with WCF as the http listener?
Is there any possibility that the focus on http_listener in REST SDK will never be picked up in future and will it get dropped out from the future plan?
Ganesha
↧
New Post: extract_json performance
Hi Ganesha,
Regarding the http_listener - we are not actively doing any work on it right now and don't currently have any plans to. It is missing some important features and hasn't received the same amount of work as other parts of the library. This is the reason it is marked as beta and in an experimental namespace.
Steve
Regarding the http_listener - we are not actively doing any work on it right now and don't currently have any plans to. It is missing some important features and hasn't received the same amount of work as other parts of the library. This is the reason it is marked as beta and in an experimental namespace.
Steve
↧
New Post: How should I handle websocket_client reconnects?
Hi DesktopMan,
Our websocket client doesn't allow reconnecting by calling the connect function again. You can simply create a new websocket client object instance and then call connect.
Also FYI it probably is dangerous to endlessly loop if an exception occurs trying to reconnect again. You probably only will want to try a couple of times and potentially not depending on the error.
Steve
Our websocket client doesn't allow reconnecting by calling the connect function again. You can simply create a new websocket client object instance and then call connect.
Also FYI it probably is dangerous to endlessly loop if an exception occurs trying to reconnect again. You probably only will want to try a couple of times and potentially not depending on the error.
Steve
↧
↧
Commented Unassigned: defect in lazy initialization of static in windows_category() [391]
The instance object in the code below races on construction during multi-threaded use of the API when generating errors. The generated code on Windows/VS2013 is not thread safe. The actual error is generated in (http_client_winhttp.cpp) build_error_message while trying to access the object via windows_category(). I have not created a minimal reproduction scenario...
Simple fix, call the function somewhere safe (i.e. single threaded) to instantiate the object:
```c++
utility::details::windows_category();
```
```c++
const std::error_category & __cdecl windows_category()
{
static details::windows_category_impl instance;
return instance;
}
```
another:
```c++
const web::json::details::json_error_category_impl& web::json::details::json_error_category()
{
static web::json::details::json_error_category_impl instance;
return instance;
}
```
Comments: Started from the BingRequest sample and modified. The issue occurs if/when multiple threads generate an error simultaneously and race in windows_category. Repro'd on 64 bit Windows 8.1. The machine is a dual-proc Xeon w/ 32 cores -- makes race conditions easier to repro... This may take minutes or hours to reproduce. Run this from a batch file in a loop until it crashes with VS or Windbg set as your default crash handler/debugger. ``` :START call foo.exe goto START ``` ```c++ #include <cpprest/http_client.h> #include <array> #include <algorithm> #include <ctime> using namespace web::http; using namespace web::http::client; using namespace concurrency::streams; int wmain(int argc, wchar_t *args[]) { CASABLANCA_UNREFERENCED_PARAMETER(argc); CASABLANCA_UNREFERENCED_PARAMETER(args); http_client_config config; http_client client(U("http://fail.dns.zzz/bucket/foo.ext"), config); std::srand(unsigned(std::time(0))); std::vector<int> a(5 + std::rand() % 50); std::cout << "Clients: " << a.size() << std::endl; Concurrency::parallel_for_each(a.cbegin(), a.cend(), [&](const int &i) { try { Concurrency::streams::container_buffer<std::vector<uint8_t>> buf; auto request = http_request{ methods::GET }; request.set_response_stream(buf.create_ostream()); auto response = client.request(request).get(); response.content_ready().get(); } catch (std::exception &ex) { // std::wcout << "exception: '" << ex.what() << std::endl; } }); return 0; } ```
Simple fix, call the function somewhere safe (i.e. single threaded) to instantiate the object:
```c++
utility::details::windows_category();
```
```c++
const std::error_category & __cdecl windows_category()
{
static details::windows_category_impl instance;
return instance;
}
```
another:
```c++
const web::json::details::json_error_category_impl& web::json::details::json_error_category()
{
static web::json::details::json_error_category_impl instance;
return instance;
}
```
Comments: Started from the BingRequest sample and modified. The issue occurs if/when multiple threads generate an error simultaneously and race in windows_category. Repro'd on 64 bit Windows 8.1. The machine is a dual-proc Xeon w/ 32 cores -- makes race conditions easier to repro... This may take minutes or hours to reproduce. Run this from a batch file in a loop until it crashes with VS or Windbg set as your default crash handler/debugger. ``` :START call foo.exe goto START ``` ```c++ #include <cpprest/http_client.h> #include <array> #include <algorithm> #include <ctime> using namespace web::http; using namespace web::http::client; using namespace concurrency::streams; int wmain(int argc, wchar_t *args[]) { CASABLANCA_UNREFERENCED_PARAMETER(argc); CASABLANCA_UNREFERENCED_PARAMETER(args); http_client_config config; http_client client(U("http://fail.dns.zzz/bucket/foo.ext"), config); std::srand(unsigned(std::time(0))); std::vector<int> a(5 + std::rand() % 50); std::cout << "Clients: " << a.size() << std::endl; Concurrency::parallel_for_each(a.cbegin(), a.cend(), [&](const int &i) { try { Concurrency::streams::container_buffer<std::vector<uint8_t>> buf; auto request = http_request{ methods::GET }; request.set_response_stream(buf.create_ostream()); auto response = client.request(request).get(); response.content_ready().get(); } catch (std::exception &ex) { // std::wcout << "exception: '" << ex.what() << std::endl; } }); return 0; } ```
↧
Edited Issue: Mismatched new/delete Windows streams file I/O - no leak, but bad code [189]
I still haven’t figured out in the non-error path who is responsible for deleting this buffer. One immediate problem is we allocate a buffer on the heap in many places like the following in fileio_winrt.cpp at line 328:
fInfo->m_buffer = new char[fInfo->m_bufsize*char_size];
Then in numerous places we mismatch a delete, for example look at the code just 4 lines above:
if ( fInfo->m_buffer != nullptr )
delete fInfo->m_buffer;
This is bad and I see this going on in lots of places.
fInfo->m_buffer = new char[fInfo->m_bufsize*char_size];
Then in numerous places we mismatch a delete, for example look at the code just 4 lines above:
if ( fInfo->m_buffer != nullptr )
delete fInfo->m_buffer;
This is bad and I see this going on in lots of places.
↧
Commented Issue: uri_exception is thrown when uri contains square brackets [382]
`uri_builder` in `windows_request_context::read_headers_io_completion` throws web::uri_exception when the uri contains square brackets.
The encode_uri method handles square brackets as general delimiters and therefor will not encode them.
The encoded uri is passed to the uri_builder that throws the web::uri_exception.
The following code snippet reproduces the exception:
```cpp
auto encodedUri = uri::encode_uri(L"http://localhost/service[1]?param[]=123¶m[]=456");
// assert(encodedUri == L"http://localhost/service%5B1%5D?param%5B%5D=123¶m%5B%5D=456");
uri_builder builder(encodedUri);
```
In compare, the JavaScript `encodeURI` method encodes the square brackets as this exampel shows:
```javascript
console.log(encodeURI("http://localhost/service[1]?param[]=123¶m[]=456"));
// -> http://localhost/service%5B1%5D?param%5B%5D=123¶m%5B%5D=456
```
Is there a reason to keep the brackets unencoded and why is the behavior different to javascript?
Comments: Hi Steve You're right, when the brackets are encoded, it works. Sorry for that! Markus
The encode_uri method handles square brackets as general delimiters and therefor will not encode them.
The encoded uri is passed to the uri_builder that throws the web::uri_exception.
The following code snippet reproduces the exception:
```cpp
auto encodedUri = uri::encode_uri(L"http://localhost/service[1]?param[]=123¶m[]=456");
// assert(encodedUri == L"http://localhost/service%5B1%5D?param%5B%5D=123¶m%5B%5D=456");
uri_builder builder(encodedUri);
```
In compare, the JavaScript `encodeURI` method encodes the square brackets as this exampel shows:
```javascript
console.log(encodeURI("http://localhost/service[1]?param[]=123¶m[]=456"));
// -> http://localhost/service%5B1%5D?param%5B%5D=123¶m%5B%5D=456
```
Is there a reason to keep the brackets unencoded and why is the behavior different to javascript?
Comments: Hi Steve You're right, when the brackets are encoded, it works. Sorry for that! Markus
↧
New Post: Regarding Security on http_listener
Thanks Steve. I am using abstract session pattern to provide security on top of webservice.
-Ram
-Ram
↧
↧
New Post: extract_json performance
Hi Steve,
I did 2 things today.
Ganesha
I did 2 things today.
- Used 2.6.0 of REST SDK instead of 2.5.0. There seems to be some fix in 2.6. for http_listener.
-
Used a different client system
The download of data is now very fast (between 1 to 1.5 sec)! So this is certainly a good news.:-)
Ganesha
↧
New Post: extract_json performance
Hi Ganesha,
A few examples of major features missing include, authentication, HTTPS (only implemented on Windows), and persistent connections (only implemented on Windows).
Steve
A few examples of major features missing include, authentication, HTTPS (only implemented on Windows), and persistent connections (only implemented on Windows).
Steve
↧
New Post: extract_json performance
Hi Steve,
I am interested only in Windows deployment. So do you mean the features in http_listener are almost complete for Windows deployment? Anything is missing for Windows?
Ganesha
I am interested only in Windows deployment. So do you mean the features in http_listener are almost complete for Windows deployment? Anything is missing for Windows?
Ganesha
↧
New Post: extract_json performance
On all the platforms I consider the http_listener to still be a beta state. Depending on what features you want or need it might be complete enough, there is no authentication support, no client or server certificate support, very few if any configuration options. You can take a look at the outstanding issues, by clicking the "Issues" tab and selecting the http_listener component. From a quality perspective we didn't do any scale testing, reliability testing (since a server component), very little performance work has been done.
You always could implement and contribute back any features if you wanted to as well.
Steve
You always could implement and contribute back any features if you wanted to as well.
Steve
↧
↧
New Post: extract_json performance
Thanks Steve. Let me check other aspects that we need.
Ganesha
Ganesha
↧
Updated Release: C++ REST SDK 2.7.0
websockets
miscellaneous
- Merged pull request allowing through CMake to use an external version of Websocket++. #294
miscellaneous
- Merged a couple of pull requests (here, here, and here) fixing several issues with arm64. #312, #291
- Merged pull request fixing double include issues when building with CMake.
- Merged pull request fixing cast issue on FreeBSD.
↧
New Post: IIS support
Hi Steve,
It looks like IIS hosting is not supported for REST SDK, right?
Is there anyway to hook it up with IIS Native module?
Ganesha
It looks like IIS hosting is not supported for REST SDK, right?
Is there anyway to hook it up with IIS Native module?
Ganesha
↧