I am using the Amazon MWS API to download a text file containing orders. 70% of the time the file downloads perfectly but sometime I get corruption. If I look at the file via the Amazon web front end it looks fine. However by the time it is stored on the local hard drive it can be corrupted. I can see no pattern as to why some file have this problem and others don't.
The corruption I am talking about looks like this (data changed but highlights the problem)
payments-status order-id order-item-id payments-date payments-transaction-id item-name listing-id sku price shipping-fee quantity-purchased total-price purchase-date batch-id buyer-email buyer-name recipient-name ship-address-1 ship-address-2 ship-city ship-state ship-zip ship-country special-comments upc ship-method sales-channel VAT
205-2599941-6954759 11274234567395 07/01/2015 15:22:48 Europe/London My Product 1 1128h56V4X 9785858599053 3.59 2.8 1 6.39 07/01/2015 15:22:48 Europe/London bwfdsfs3tmf15w@marketplace.amazon.co.uk Mr F Bloggs Mr Fred Bloggs My Cottage XXXXXXX AnyTown Isle of Man XX1 2XX GB standard
205-499978-7575554 68198765457651 07/01/2015 15:28:23 Europe/London My Product 2 1128h56MWN9 9785858524633 5.99 2.8 1 8.79 07/01/2015 15:28:23 Europe/London zyrre44rrr7nk@marketplace.amazon.co.uk Jane Bloggs Prof Jane Blogs Her Cottage, XXXXXXX AnyTown Surrey XX1 2XX GB standard
.99 2.8 1 8.79 07/01/2015 14:07:33 Europe/London nuy76sz9fw4lzk@marketplace.amazon.co.uk Joe Bloggs Joe Bloggs 71 The Houses AnyTown MyCounty AA1 1WW GB standard
026-199993-8483529 38758765432635 07/01/2015 14:07:05 Europe/London My Product 3 ... 118h56QMBQR 9781783610433 5.99 2.8 1 8.79 07/01/2015 14:07:05 Europe/London h0hx8werwew4t4@marketplace.amazon.co.uk Claire Blogs Claire Bloggs XXXXXXX AnyTown JERSEY Channel Islands XX1 2XX GB standard
202-788880-2669961 46830987654327 07/01/2015 14:22:21 Europe/London My Product 4 1128h56V89 978585856766 3.59 2.8 1 6.39 07/01/2015 14:22:21 Europe/London w608ewrwewrk74l@marketplace.amazon.co.uk Harry Blogs Harry Bloggs XXXXXXX AnyTown Co Durham XX1 2XX GB standard
203-355556-3369905 34856789098939 07/01/2015 14:24:42 Europe/London My Product 5 118h56QLX1L 9785858512400 5.99 2.8 1 8.79 07/01/2015 14:24:42 Europe/London zgw0rwewere98y@marketplace.amazon.co.uk Aimee Blogs Aimee Bloggs XXXXXXX 25 Any Street AnyTown XX1 2XX GB standard
204-4888812-4296349 14153456784747 07/01/2015 14:41:10 Europe/London My Product 6 11278h5665F 9785858546360 3.59 2.8 1 6.39 07/01/2015 14:41:10 Europe/London h5mcwerwewwwfvrg3@marketplace.amazon.co.uk Julie Bloggs Julie Bloggs XXXXXXX AnyTown Devon XX1 2XX GB standard
026-1666655-8765
This is a tab delimited file and the first column is always a blank field. The first two lines show correct data and in a real file there are plenty of correct lines before I see this happen. The 3rd line is missing a number of columns from the start of the line and the last line is missing part of the second field and all the rest of the columns.
I see this happening mainly towards the end of the file.
So the code I use is the library straight from the Amazon site Reports C# API:
private T Invoke<T, K>(IDictionary<String, String> parameters, K clazz)
{
String actionName = parameters["Action"];
T response = default(T);
String responseBody = null;
HttpStatusCode statusCode = default(HttpStatusCode);
ResponseHeaderMetadata rhm = null;
// Verify service URL is set.
if (String.IsNullOrEmpty(config.ServiceURL))
{
throw new MarketplaceWebServiceException(new ArgumentException(
"Missing serviceUrl configuration value. You may obtain a list of valid MWS URLs by consulting the MWS Developer's Guide, or reviewing the sample code published along side this library."));
}
/* Add required request parameters */
AddRequiredParameters(parameters);
String queryString = GetParametersAsString(parameters);
byte[] requestData = new UTF8Encoding().GetBytes(queryString);
HttpWebRequest request;
bool isStreamingResponse = ExpectStreamingResponse(typeof(K));
bool shouldRetry = true;
int retries = 0;
do
{
/* Submit the request and read response body */
try
{
RequestType requestType = GetMarketplaceWebServiceRequestType(typeof(K));
switch (requestType)
{
case RequestType.STREAMING:
{
SubmitFeedRequest req = clazz as SubmitFeedRequest;
if (req != null)
{
// SubmitFeedRequests can configure the content type.
request = ConfigureWebRequest(queryString, req.ContentType);
}
else
{
// Send request using a default content-type.
request = ConfigureWebRequest(queryString, new ContentType(MediaType.OctetStream));
}
}
break;
default:
request = ConfigureWebRequest(requestData.Length);
break;
}
WebHeaderCollection headers = request.Headers;
IDictionary<String, String> headerMap = GetHttpHeaderValues(clazz);
foreach (String key in headerMap.Keys)
{
headers.Add(key, headerMap[key]);
}
using (Stream requestStream = request.GetRequestStream())
{
switch (requestType)
{
case RequestType.STREAMING:
Stream inputStream = GetTransferStream(clazz, StreamType.REQUEST_STREAM);
inputStream.Position = 0;
CopyStream(inputStream, requestStream);
break;
default:
requestStream.Write(requestData, 0, requestData.Length);
break;
}
requestStream.Close();
}
using (HttpWebResponse httpResponse = request.GetResponse() as HttpWebResponse)
{
statusCode = httpResponse.StatusCode;
rhm = new ResponseHeaderMetadata(
httpResponse.GetResponseHeader("x-mws-request-id"),
httpResponse.GetResponseHeader("x-mws-response-context"),
httpResponse.GetResponseHeader("x-mws-timestamp"));
if (isStreamingResponse && statusCode == HttpStatusCode.OK)
{
response = HandleStreamingResponse<T>(httpResponse, clazz);
}
else
{
StreamReader reader = new StreamReader(httpResponse.GetResponseStream(), Encoding.UTF8);
responseBody = reader.ReadToEnd();
XmlSerializer serlizer = new XmlSerializer(typeof(T));
response = (T)serlizer.Deserialize(new StringReader(responseBody));
}
PropertyInfo pi = typeof(T).GetProperty("ResponseHeaderMetadata");
pi.SetValue(response, rhm, null);
shouldRetry = false;
}
/* Attempt to deserialize response into <Action> Response type */
}
/* Web exception is thrown on unsucessful responses */
catch (WebException we)
{
shouldRetry = false;
using (HttpWebResponse httpErrorResponse = (HttpWebResponse)we.Response as HttpWebResponse)
{
if (httpErrorResponse == null)
{
throw new MarketplaceWebServiceException(we);
}
statusCode = httpErrorResponse.StatusCode;
StreamReader reader = new StreamReader(httpErrorResponse.GetResponseStream(), Encoding.UTF8);
responseBody = reader.ReadToEnd();
}
/* Attempt to deserialize response into ErrorResponse type */
try
{
XmlSerializer serlizer = new XmlSerializer(typeof(ErrorResponse));
ErrorResponse errorResponse = (ErrorResponse)serlizer.Deserialize(new StringReader(responseBody));
Error error = errorResponse.Error[0];
bool retriableError = (statusCode == HttpStatusCode.InternalServerError || statusCode == HttpStatusCode.ServiceUnavailable);
retriableError = retriableError && error.Code != "RequestThrottled";
if (retriableError && retries < config.MaxErrorRetry)
{
PauseOnRetry(++retries);
shouldRetry = true;
continue;
}
else
{
shouldRetry = false;
}
/* Throw formatted exception with information available from the error response */
throw new MarketplaceWebServiceException(
error.Message,
statusCode,
error.Code,
error.Type,
errorResponse.RequestId,
errorResponse.ToXML(),
rhm);
}
/* Rethrow on deserializer error */
catch (Exception e)
{
if (e is MarketplaceWebServiceException)
{
throw e;
}
else
{
MarketplaceWebServiceException se = ReportAnyErrors(responseBody, statusCode, e, rhm);
throw se;
}
}
}
/* Catch other exceptions, attempt to convert to formatted exception,
* else rethrow wrapped exception */
catch (Exception e)
{
throw new MarketplaceWebServiceException(e);
}
} while (shouldRetry);
return response;
}
These are the values being passed in:
- parameters Count = 3 System.Collections.Generic.IDictionary<string,string> {System.Collections.Generic.Dictionary<string,string>}
+ [0] {[Action, GetReport]} System.Collections.Generic.KeyValuePair<string,string>
+ [1] {[Merchant, A999AAAA7KT4OI]} System.Collections.Generic.KeyValuePair<string,string>
+ [2] {[ReportId, 59999999994]} System.Collections.Generic.KeyValuePair<string,string>
- clazz {MyCompany.MyApp.Amazon.MarketplaceWebService.Model.GetReportRequest} MyCompany.MyApp.Amazon.MarketplaceWebService.Model.GetReportRequest
Marketplace null string
marketplaceField null string
Merchant "A999AAAA7KT4OI" string
merchantField "A999AAAA7KT4OI" string
MWSAuthToken null string
mwsAuthTokenField null string
+ Report {System.IO.FileStream} System.IO.Stream {System.IO.FileStream}
+ report {System.IO.FileStream} System.IO.Stream {System.IO.FileStream}
ReportId "59999999994" string
reportIdField "59999999994" string
The resulting file is saved to disk.
When this problem first presented itself I tried to code a workaround while I looked for a reason and the correct fix. This is to open the file and loop through it line by line to ensure that each line start with a tab followed by 3 digits and a hyphen but now this doesn't work because of the last line. Now I have a bit of time to try and get to the bottom of why this is happening.
I raised it with Amazon but they say that the problem is with my code (even though it is theirs) not parsing the file correctly.
Can anyone suggest why this is happening or point me in a direction I should be looking?
Aucun commentaire:
Enregistrer un commentaire