API Questions

[SherpDaWerp]

Hello

I'm in the process of making an api-based tool so I can keep my bots active and answering issues without logging in to all 6 of them. Due to my personal knowledge of PHP, I'm keen to continue using it instead of learning Lua or cURL just for this one task.

How do I provide a password to the API when I connect via PHP? Currently I'm trying to connect to something like this:

Code: Select all

https://www.nationstates.net/cgi-bin/api.cgi?a=verify&nation=nation_name&password=password&q=issues

but all I get is authentication errors. Do any of the mods (or anyone really) know how I can provide a password to the api while still connecting via a URL? Or should I just bite the bullet and go with another language...

[Trotterdam]

As explained in the API documentation, passwords need to be submitted in the HTTP header, as "X-Password".

I have no experience with PHP myself, but according to the manual, it looks like you probably need something like this:

Code: Select all

stream_context_create(array('http' => array('header'  => 'X-Password: password')))

By the way, if you don't know how to handle HTTP headers, are you setting your user agent properly?

...Also, under what use case do you need to combine a=verify with a private shard? The point of the verification API is to verify ownership of nations that aren't yours. If you already know the nation's password, that's redundant.

[SherpDaWerp]

I've set the user agent, it's just a

Code: Select all

ini_set("user_agent",$useragent);

...Also, under what use case do you need to combine a=verify with a private shard? The point of the verification API is to verify ownership of nations that aren't yours. If you already know the nation's password, that's redundant.

I know the password, but I can't access the private shards in the API (i.e. issues) without verifying that I own my puppets. Unless I'm very much mistaken, that involves giving the API your puppet's name and then password so it can login and verify you as actually owning that puppet.

As explained in the API documentation, passwords need to be submitted in the HTTP header, as "X-Password".

I tried using

Code: Select all

https://www.nationstates.net/cgi-bin/api.cgi?a=verify&nation=nation_name&x-password=password&q=issues

to verify x-password, but that didn't work either. I'll try your approach and see if that works.

[SherpDaWerp]

I know this is a mad backflip, but I found out the php software I use has curl already installed, so I bit the bullet and gave it a go. My code is now

Code: Select all

<?php
$useragent = me;
$nation = nation_name;
$url = "https://www.nationstates.net/cgi-bin/api.cgi?nation=".$nation."&q=issues";

$curl = curl_init();
curl_setopt_array($curl,
   array(
      CURLOPT_URL      => $url,
      CURLOPT_HEADER   => "X-Password: password",
      CURLOPT_USERAGENT   => me
   )
);

$data = curl_exec($curl);
curl_close($curl);
echo $data;
?>

This should be setting the header to include X-Password: password, as mentioned in the documentation. But it still returns 403 error and asks for credentials. Any suggestions?

[Frisbeeteria]

I've set the user agent, it's just a

Code: Select all

ini_set("user_agent",$useragent);

The user agent is supposed to be your main nation name or email address so we can contact you if there's a problem or your script goes nuts. Making a generic ("user_agent",$useragent) tells us nothing when we look at the API logs. Fix that.

[Phydios]

I know this is a mad backflip, but I found out the php software I use has curl already installed, so I bit the bullet and gave it a go. My code is now

Code: Select all

<?php
$useragent = me;
$nation = nation_name;
$url = "https://www.nationstates.net/cgi-bin/api.cgi?nation=".$nation."&q=issues";

$curl = curl_init();
curl_setopt_array($curl,
   array(
      CURLOPT_URL      => $url,
      CURLOPT_HEADER   => "X-Password: password",
      CURLOPT_USERAGENT   => me
   )
);

$data = curl_exec($curl);
curl_close($curl);
echo $data;
?>

This should be setting the header to include X-Password: password, as mentioned in the documentation. But it still returns 403 error and asks for credentials. Any suggestions?

Sorry for the stupid question, but you aren't literally sending "X-Password: password" in the header, right? You're sending your actual password?

[SherpDaWerp]

I've set the user agent, it's just a

Code: Select all

ini_set("user_agent",$useragent);

The user agent is supposed to be your main nation name or email address so we can contact you if there's a problem or your script goes nuts. Making a generic ("user_agent",$useragent) tells us nothing when we look at the API logs. Fix that.

I set the useragent to my email address in the actual code I run, *email redacted*, but I just removed it along with my password and nation name in favor of generic variables for the version I sent to the forum.

Sorry for the stupid question, but you aren't literally sending "X-Password: password" in the header, right? You're sending your actual password?

From the api docs:

The first time you access a private shard, you will probably provide X-Password, since that's all you know. It's fine to keep doing this for all requests in some circumstances, such as if you are running your own script on your own computer, which only makes one request every so often.

Once this works, I plan to use the X-pin it (should) give me for further requests so there's less load on the api.

EDIT: At the moment, this is my exact code, but with my nation name and password blocked out:

Code: Select all

<?php
$useragent = '*email redacted*';
$nation = '*****';
$password = '*****';
$option = 'issues';

$url = 'https://www.nationstates.net/cgi-bin/api.cgi?q='.$option;

$query = curl_init();
curl_setopt_array($query,
   array(
      CURLOPT_URL            => $url,
      CURLOPT_HEADER         => 'X-Password: '.$password,
      CURLOPT_HEADER         => 'nation: '.$nation,
      CURLOPT_USERAGENT      => $useragent,
      CURLOPT_RETURNTRANSFER   => true,
   )
);

$data = curl_exec($query);

if($data === FALSE) {
    die(curl_error($query));
} else {
   curl_close($query);
}

echo '<code>a '.$data.' a</code>';
?>

It returns "a a", which indicates the $data variable is empty, which in turn says there's something wrong with the request somewhere, but without a concrete error message I don't know what...

[Phydios]

Sounds like you need to see any error messages left by curl_exec. This StackOverflow page seems like it would help, though I have no experience with curl myself.

[Trotterdam]

It returns "a a", which indicates the $data variable is empty, which in turn says there's something wrong with the request somewhere, but without a concrete error message I don't know what...

Only the password and user agent belong in the header. This part doesn't:

CURLOPT_HEADER => 'nation: '.$nation,

Continue doing that the way you've been doing before.

Oh yeah, also:

Once this works, I plan to use the X-pin it (should) give me for further requests so there's less load on the api.

The X-Pin is only valid for a single session, and becomes invalidated if your script idles for too long or if you log into the nation normally in your browser. It's useful if your script needs to perform several private shards/commands in rapid succession, but make sure you still keep the X-Password part as well, and that any X-Pin you use should actually be grabbed in the script itself rather than just hardcoding what you got the first time you tested it (as the flowchart in the manual shows, if you send both X-Password and X-Pin in the same request, then the X-Password will be ignored if the X-Pin is valid, but will be used as a fallback if the X-Pin is invalid). If your script doesn't need to perform more than one query between long waits, it's not worth bothering with the X-Pin at all.

[All Wild Things]

Are you including the nation in your URL?

eg

https://www.nationstates.net/cgi-bin/ap ... a&q=issues

[SherpDaWerp]

Are you including the nation in your URL?

eg

https://www.nationstates.net/cgi-bin/ap ... a&q=issues

I've tried both with and without. If I supply the nation in the url, it gives me a 403 error: incorrect credentials. (I know I have the password and nation name correct so this is not the case.)
If I supply the nation in the cURL array, using CURLOPT_POST, it returns nothing, as described above.

[Trotterdam]

I've tried both with and without. If I supply the nation in the url, it gives me a 403 error: incorrect credentials. (I know I have the password and nation name correct so this is not the case.)

That's definitely how you're supposed to do it, so something is wrong with your cURL code.

I had a look at the manual, and it looks like you're using CURLOPT_HEADER incorrectly. That is a boolean value used to indicate whether you want to retrieve the header that the server returns (which you need to obtain the X-Pin). To set the header you send to the server, you need:

Code: Select all

CURLOPT_HTTPHEADER => array('X-Password: password')

[SherpDaWerp]

I've tried both with and without. If I supply the nation in the url, it gives me a 403 error: incorrect credentials. (I know I have the password and nation name correct so this is not the case.)

That's definitely how you're supposed to do it, so something is wrong with your cURL code.

I had a look at the manual, and it looks like you're using CURLOPT_HEADER incorrectly. That is a boolean value used to indicate whether you want to retrieve the header that the server returns (which you need to obtain the X-Pin). To set the header you send to the server, you need:

Code: Select all

CURLOPT_HTTPHEADER => array('X-Password: password')

Yeah, that was it. Thanks everyone for the help, and for reference my working code is now:

Code: Select all

<?php
$useragent = me;
$nation = nation_name;
$password = password;
$option = 'issues';

$url = 'https://www.nationstates.net/cgi-bin/api.cgi?nation='.$nation.'&q='.$option;

$query = curl_init();
curl_setopt_array($query,
   array(
      CURLOPT_URL            => $url,
      CURLOPT_POST         => 'nation='.$nation.'',
      CURLOPT_HTTPHEADER      => array('X-Password: '.$password.'', 'nation: '.$nation.''),
      CURLOPT_USERAGENT      => $useragent,
      CURLOPT_RETURNTRANSFER   => true,
      CURLOPT_SSL_VERIFYHOST   => false,
      CURLOPT_SSL_VERIFYPEER   => false
   )
);

$data = curl_exec($query);
$error = curl_error($query);

print '<code>. '.$error.' . '.$data.' . '.$query.' .</code>';

curl_close($query);
?>


[Trotterdam]

$url = 'https://www.nationstates.net/cgi-bin/api.cgi?nation='.$nation.'&q='.$option;

CURLOPT_POST => 'nation='.$nation.'',

These are redundant. You're submitting the same information as both GET and POST data. It does no harm, I suppose, but it's also pointless (and risks introducing bugs if you later update your script but forget that you're doing the essentially same thing in two different places and only update one). Pick one and stick to it (99% of web applications don't pay attention to the difference, the NS API included, so either one will work).

And including the nation name in the header is completely pointless, it doesn't get looked at.

EDIT: Actually, you're also doing the POST data wrong. Again, CURLOPT_POST is a boolean saying whether to use the POST method instead of the GET method. The actual data to be submitted with the POST method belongs in CURLOPT_POSTFIELDS. So I guess your CURLOPT_POST line is actually not doing anything since you're using it incorrectly, and your script is just using the GET method. Just delete that line if you don't want to worry about it.

[SherpDaWerp]

These are redundant.

And including the nation name in the header is completely pointless, it doesn't get looked at.

EDIT: Actually, you're also doing the POST data wrong. Again, CURLOPT_POST is a boolean saying whether to use the POST method instead of the GET method. The actual data to be submitted with the POST method belongs in CURLOPT_POSTFIELDS. So I guess your CURLOPT_POST line is actually not doing anything since you're using it incorrectly, and your script is just using the GET method. Just delete that line if you don't want to worry about it.

Fixed, thanks! Though sending the 'answer issue' command is now playing up .

EDIT: I'm getting a 400 (Bad Request), sending the following cURL code:

Code: Select all

$post = array(
  'nation'  => $current_nation['nation'],
  'c'  => 'issue',
  'issue'  => $first_issue_number,
  'option'  => $option
);

curl_setopt_array($query,
  array(
    CURLOPT_URL               => 'https://www.nationstates.net/cgi-bin/api.cgi?',
    CURLOPT_HTTPHEADER         => array('X-Pin: '.$current_nation[0]['xpin'], 'X-Password: '.$current_nation['password']),
    CURLOPT_USERAGENT         => $useragent,
    CURLOPT_POST               => 'POST',
    CURLOPT_POSTFIELDS         => $post,
    CURLOPT_HEADER            => true,
    CURLOPT_VERBOSE            => true,
    CURLOPT_RETURNTRANSFER      => true,
    CURLOPT_SSL_VERIFYHOST      => false,
    CURLOPT_SSL_VERIFYPEER      => false
  )
);


This is the way the API docs do it, sending the data separately by POST-ing it after the URL. I've also tried sending it at the end of the URL, but that doesn't work either.

[Trotterdam]

EDIT: I'm getting a 400 (Bad Request), sending the following cURL code:

You need

Code: Select all

    CURLOPT_POST               => true,


not

Code: Select all

    CURLOPT_POST               => 'POST',

Although this says that most strings get treated as "true" anyway, so I'm not sure what's going on.

Also try taking the question mark off the end of the CURLOPT_URL. It probably does no harm, but it's also pointless if you're not passing any data through it.

EDIT: Wait, there's this:

Passing an array to CURLOPT_POSTFIELDS will encode the data as multipart/form-data, while passing a URL-encoded string will encode the data as application/x-www-form-urlencoded.

I suspect the server might not understand multipart/form-data. If this is the case, you will need to change

Code: Select all

$post = array(
  'nation'  => $current_nation['nation'],
  'c'  => 'issue',
  'issue'  => $first_issue_number,
  'option'  => $option
);

to

Code: Select all

$post = 'nation='.$current_nation['nation']
      .'&c=issue'
      .'&issue='.$first_issue_number
      .'&option='.$option;

(Disclaimer: I have zero experience with PHP and have not tested this, I THINK this is the right syntax but I'm just eyeballing it.)

The server definitely accepts POST data in application/x-www-form-urlencoded format, though. That's what I use.

[Trotterdam]

So, did you ever get this working?

I need to know if I was helpful. My honor depends on it

[SherpDaWerp]

Not quite yet, I'm still having troubles with answering issues. The lack of updates is because school suddenly hit me with 2 assignments, 2 exams and a ski trip, so this got put on the back-burner. Anyway, I'll put my code here (and remove identifying information), so you can go through it if you want. It *should* be working, but there's gonna be one comma out of place or something mucking it all up...

Code: Select all

<?php
   $useragent = ' ***** ';
   $option = '1';
   set_time_limit(0);

   header("Content-Type:text/plain; charset=UTF-8");

   $current_nation = array(
      'nation'   => ' ****** ',
      'password'   => ' ***** ',
      'xpin'      => ' '
   );

   $query = curl_init();
   curl_setopt_array($query,
      array(
         CURLOPT_URL            => 'https://www.nationstates.net/cgi-bin/api.cgi?nation='.$current_nation['nation'].'&q=issues',
         CURLOPT_HTTPHEADER      => array('X-Password: '.$current_nation['password']),
         CURLOPT_USERAGENT      => $useragent,
         CURLOPT_HEADER         => true,
         CURLOPT_VERBOSE         => true,
         CURLOPT_RETURNTRANSFER   => true,
         CURLOPT_SSL_VERIFYHOST   => false,
         CURLOPT_SSL_VERIFYPEER   => false
      )
   );

   $data = curl_exec($query);
   $header_size = curl_getinfo($query, CURLINFO_HEADER_SIZE);
   $header = substr($data, 0, $header_size);
   $body = substr($data, $header_size);
   curl_close($query);
   
   echo $data;

   sleep(5);

   // /*
   $split1 = explode("x-pin: ", $header);
   $split2 = explode("x-ratelimit-requests-seen:", $split1[1]);
   $current_nation['xpin'] = $split2[0];

   $split3 = explode('<ISSUE id="', $body);
   if ($split3[1]) {
      $split4 = explode('">', $split3[1]);
      $first_issue_number = $split4[0];

      $query = curl_init();
      curl_setopt_array($query,
         array(
            CURLOPT_URL            => 'https://www.nationstates.net/cgi-bin/api.cgination='.$current_nation['nation'].'&c=issues&issue='.$first_issue_number.'&option='.$option,
            CURLOPT_HTTPHEADER      => array('X-Pin: '.$current_nation['xpin'], 'X-Password: '.$current_nation['password']), //, 'Content-Type: application/x-www-form-urlencoded'
            CURLOPT_USERAGENT      => $useragent,
            CURLOPT_POST         => true,
            CURLOPT_POSTFIELDS      => 'nation='.$current_nation['nation'].'&c=issue&issue='.$first_issue_number.'&option='.$option,
            CURLOPT_HEADER         => true,
            CURLOPT_VERBOSE         => true,
            CURLOPT_RETURNTRANSFER   => true,
            CURLOPT_SSL_VERIFYHOST   => false,
            CURLOPT_SSL_VERIFYPEER   => false
         )
      );
      $data = curl_exec($query);
      curl_close($query);
      echo 'url - ';
      echo 'https://www.nationstates.net/cgi-bin/api.cgination='.$current_nation['nation'].'&c=issues&issue='.$first_issue_number.'&option='.$option;
      echo ' query - ';
      echo $query;
      echo ' xpin - ';
      echo $current_nation['xpin'];
      echo ' issuenum - ';
      echo $first_issue_number;
      echo ' ------------------------------------------------ returns - ';
      echo $data;
   } else {
      echo 'No More Issues';
   }
?>


If it helps satisfy your honor, you've been very helpful

[Trotterdam]

Anyway, I'll put my code here (and remove identifying information),

You missed one.

It *should* be working, but there's gonna be one comma out of place or something mucking it all up...

Question mark, actually.

CURLOPT_URL => 'https://www.nationstates.net/cgi-bin/api.cgination='.$current_nation['nation'].'&c=issues&issue='.$first_issue_number.'&option='.$option,

That would be your problem.

Seriously, just delete tthat bit. There's no need to send the same data through both GET and POST. You can try one of the other doesn't work, but don't do both at the same time.

[SherpDaWerp]

You missed one.

So I did. At least it wasn't a password...

It *should* be working, but there's gonna be one comma out of place or something mucking it all up...

Question mark, actually.

CURLOPT_URL => 'https://www.nationstates.net/cgi-bin/api.cgination='.$current_nation['nation'].'&c=issues&issue='.$first_issue_number.'&option='.$option,

That would be your problem.

Seriously, just delete tthat bit. There's no need to send the same data through both GET and POST. You can try one of the other doesn't work, but don't do both at the same time.

I am aware of that question mark thingy, and whether it was "/api.cgi?nation=..." or "/api.cgination=..." has no effect on the output. Even removing the stuff from the end of the url like so:

Code: Select all

curl_setopt_array($query,
   array(
      CURLOPT_URL            => 'https://www.nationstates.net/cgi-bin/api.cgi',
      CURLOPT_HTTPHEADER      => array('X-Pin: '.$current_nation['xpin'], 'X-Password: '.$current_nation['password']),
      CURLOPT_USERAGENT      => $useragent,
      CURLOPT_POST         => true,
      CURLOPT_POSTFIELDS      => 'nation='.$current_nation['nation'].'&c=issue&issue='.$first_issue_number.'&option='.$option,
      CURLOPT_HEADER         => true,
      CURLOPT_VERBOSE         => true,
      CURLOPT_RETURNTRANSFER   => true,
      CURLOPT_SSL_VERIFYHOST   => false,
      CURLOPT_SSL_VERIFYPEER   => false
   )
);

does nothing.

EDIT: I've also tried urlencoding the string that's in CURLOPT_POSTFIELDS, or passing it as an array using http_build_query. Neither of those worked:

Code: Select all

$postfield = array(
   'nation'   =>   $current_nation['nation'],
   'c'         =>   'issue',
   'issue'      =>   $first_issue_number,
   'option'   =>   $option
);
$postfields = http_build_query($postfield);

$query = curl_init();
curl_setopt_array($query,
   array(
      CURLOPT_URL            => 'https://www.nationstates.net/cgi-bin/api.cgi',
      CURLOPT_HTTPHEADER      => array('X-Pin: '.$current_nation['xpin'], 'X-Password: '.$current_nation['password']),
      CURLOPT_USERAGENT      => $useragent,
      CURLOPT_POST         => true,
      CURLOPT_POSTFIELDS      => $postfields,
      CURLOPT_HEADER         => true,
      CURLOPT_VERBOSE         => true,
      CURLOPT_RETURNTRANSFER   => true,
      CURLOPT_SSL_VERIFYHOST   => false,
      CURLOPT_SSL_VERIFYPEER   => false
   )
);

[Trotterdam]

Okay, I'm stuck That should work.

I guess I should try using an API command myself sometime to see if it works when I do it.

Just to be clear, the first half, does work, but the second doesn't? Are you still getting a 400 error code?

[SherpDaWerp]

Okay, I'm stuck That should work.

I guess I should try using an API command myself sometime to see if it works when I do it.

Just to be clear, the first half, does work, but the second doesn't? Are you still getting a 400 error code?

The first half works fine, no issues there. It returns a list of all the available issues, their descriptions, numbers, authors, all the usual stuff.
The second command, however, returns nothing (not even an error code), and has no effect on the nation in question. It's certainly very weird...

[Trotterdam]

I guess I should try using an API command myself sometime to see if it works when I do it.

So, I just tried it (though not with your code, since I've never used PHP) and it works fine for me. I also confirmed that both GET and POST methods work, even for commands.

Incidentally, how are you checking for HTTP errors? I'm not seeing you do curl_getinfo($query, CURLINFO_RESPONSE_CODE), but you did mention receiving some error codes before even if you aren't now, so...

Maybe try changing your code to make a different request as your second request (maybe even a duplicate of your first request), and see if you still get the first request working but the second not.