1 // FIXME: if an exception is thrown, we shouldn't necessarily cache... 2 // FIXME: there's some annoying duplication of code in the various versioned mains 3 4 // FIXME: cgi per-request arena allocator 5 6 // i need to add a bunch of type templates for validations... mayne @NotNull or NotNull! 7 8 // FIXME: I might make a cgi proxy class which can change things; the underlying one is still immutable 9 // but the later one can edit and simplify the api. You'd have to use the subclass tho! 10 11 /* 12 void foo(int f, @("test") string s) {} 13 14 void main() { 15 static if(is(typeof(foo) Params == __parameters)) 16 //pragma(msg, __traits(getAttributes, Params[0])); 17 pragma(msg, __traits(getAttributes, Params[1..2])); 18 else 19 pragma(msg, "fail"); 20 } 21 */ 22 23 // Note: spawn-fcgi can help with fastcgi on nginx 24 25 // FIXME: to do: add openssl optionally 26 // make sure embedded_httpd doesn't send two answers if one writes() then dies 27 28 // future direction: websocket as a separate process that you can sendfile to for an async passoff of those long-lived connections 29 30 /* 31 Session manager process: it spawns a new process, passing a 32 command line argument, to just be a little key/value store 33 of some serializable struct. On Windows, it CreateProcess. 34 On Linux, it can just fork or maybe fork/exec. The session 35 key is in a cookie. 36 37 Server-side event process: spawns an async manager. You can 38 push stuff out to channel ids and the clients listen to it. 39 40 websocket process: spawns an async handler. They can talk to 41 each other or get info from a cgi request. 42 43 Tempting to put web.d 2.0 in here. It would: 44 * map urls and form generation to functions 45 * have data presentation magic 46 * do the skeleton stuff like 1.0 47 * auto-cache generated stuff in files (at least if pure?) 48 * introspect functions in json for consumers 49 50 51 https://linux.die.net/man/3/posix_spawn 52 */ 53 54 /++ 55 Provides a uniform server-side API for CGI, FastCGI, SCGI, and HTTP web applications. 56 57 --- 58 import arsd.cgi; 59 60 // Instead of writing your own main(), you should write a function 61 // that takes a Cgi param, and use mixin GenericMain 62 // for maximum compatibility with different web servers. 63 void hello(Cgi cgi) { 64 cgi.setResponseContentType("text/plain"); 65 66 if("name" in cgi.get) 67 cgi.write("Hello, " ~ cgi.get["name"]); 68 else 69 cgi.write("Hello, world!"); 70 } 71 72 mixin GenericMain!hello; 73 --- 74 75 Test on console (works in any interface mode): 76 $(CONSOLE 77 $ ./cgi_hello GET / name=whatever 78 ) 79 80 If using http version (default on `dub` builds, or on custom builds when passing `-version=embedded_httpd` to dmd): 81 $(CONSOLE 82 $ ./cgi_hello --port 8080 83 # now you can go to http://localhost:8080/?name=whatever 84 ) 85 86 87 Compile_versions: 88 89 If you are using `dub`, use: 90 91 ```sdlang 92 subConfiguration "arsd-official:cgi" "VALUE_HERE" 93 ``` 94 95 or to dub.json: 96 97 ```json 98 "subConfigurations": {"arsd-official:cgi": "VALUE_HERE"} 99 ``` 100 101 to change versions. The possible options for `VALUE_HERE` are: 102 103 $(LIST 104 * `embedded_httpd` for the embedded httpd version (built-in web server). This is the default. 105 * `cgi` for traditional cgi binaries. 106 * `fastcgi` for FastCGI builds. 107 * `scgi` for SCGI builds. 108 ) 109 110 With dmd, use: 111 112 $(TABLE_ROWS 113 114 * + Interfaces 115 + (mutually exclusive) 116 117 * - `-version=plain_cgi` 118 - The default building the module alone without dub - a traditional, plain CGI executable will be generated. 119 * - `-version=embedded_httpd` 120 - A HTTP server will be embedded in the generated executable. This is default when building with dub. 121 * - `-version=fastcgi` 122 - A FastCGI executable will be generated. 123 * - `-version=scgi` 124 - A SCGI (SimpleCGI) executable will be generated. 125 126 * - `-version=embedded_httpd_threads` 127 - The embedded HTTP server will use a single process with a thread pool. (use instead of plain `embedded_httpd` if you want this specific implementation) 128 * - `-version=embedded_httpd_processes` 129 - The embedded HTTP server will use a prefork style process pool. (use instead of plain `embedded_httpd` if you want this specific implementation) 130 * - `-version=embedded_httpd_processes_accept_after_fork` 131 - It will call accept() in each child process, after forking. This is currently the only option, though I am experimenting with other ideas. You probably should NOT specify this right now. 132 133 * + Tweaks 134 + (can be used together with others) 135 136 * - `-version=cgi_with_websocket` 137 - The CGI class has websocket server support. 138 139 * - `-version=with_openssl` 140 - not currently used 141 * - `-version=cgi_embedded_sessions` 142 - The session server will be embedded in the cgi.d server process 143 * - `-version=cgi_session_server_process` 144 - The session will be provided in a separate process, provided by cgi.d. 145 ) 146 147 Compile_and_run: 148 149 For CGI, `dmd yourfile.d cgi.d` then put the executable in your cgi-bin directory. 150 151 For FastCGI: `dmd yourfile.d cgi.d -version=fastcgi` and run it. spawn-fcgi helps on nginx. You can put the file in the directory for Apache. On IIS, run it with a port on the command line. 152 153 For SCGI: `dmd yourfile.d cgi.d -version=scgi` and run the executable, providing a port number on the command line. 154 155 For an embedded HTTP server, run `dmd yourfile.d cgi.d -version=embedded_httpd` and run the generated program. It listens on port 8085 by default. You can change this on the command line with the --port option when running your program. 156 157 You can also simulate a request by passing parameters on the command line, like: 158 159 $(CONSOLE 160 ./yourprogram GET / name=adr 161 ) 162 163 And it will print the result to stdout. 164 165 CGI_Setup_tips: 166 167 On Apache, you may do `SetHandler cgi-script` in your `.htaccess` file. 168 169 Integration_tips: 170 171 cgi.d works well with dom.d for generating html. You may also use web.d for other utilities and automatic api wrapping. 172 173 dom.d usage: 174 175 --- 176 import arsd.cgi; 177 import arsd.dom; 178 179 void hello_dom(Cgi cgi) { 180 auto document = new Document(); 181 182 static import std.file; 183 // parse the file in strict mode, requiring it to be well-formed UTF-8 XHTML 184 // (You'll appreciate this if you've ever had to deal with a missing </div> 185 // or something in a php or erb template before that would randomly mess up 186 // the output in your browser. Just check it and throw an exception early!) 187 // 188 // You could also hard-code a template or load one at compile time with an 189 // import expression, but you might appreciate making it a regular file 190 // because that means it can be more easily edited by the frontend team and 191 // they can see their changes without needing to recompile the program. 192 // 193 // Note on CTFE: if you do choose to load a static file at compile time, 194 // you *can* parse it in CTFE using enum, which will cause it to throw at 195 // compile time, which is kinda cool too. Be careful in modifying that document, 196 // though, as it will be a static instance. You might want to clone on on demand, 197 // or perhaps modify it lazily as you print it out. (Try element.tree, it returns 198 // a range of elements which you could send through std.algorithm functions. But 199 // since my selector implementation doesn't work on that level yet, you'll find that 200 // harder to use. Of course, you could make a static list of matching elements and 201 // then use a simple e is e2 predicate... :) ) 202 document.parseUtf8(std.file.read("your_template.html"), true, true); 203 204 // fill in data using DOM functions, so placing it is in the hands of HTML 205 // and it will be properly encoded as text too. 206 // 207 // Plain html templates can't run server side logic, but I think that's a 208 // good thing - it keeps them simple. You may choose to extend the html, 209 // but I think it is best to try to stick to standard elements and fill them 210 // in with requested data with IDs or class names. A further benefit of 211 // this is the designer can also highlight data based on sources in the CSS. 212 // 213 // However, all of dom.d is available, so you can format your data however 214 // you like. You can do partial templates with innerHTML too, or perhaps better, 215 // injecting cloned nodes from a partial document. 216 // 217 // There's a lot of possibilities. 218 document["#name"].innerText = cgi.request("name", "default name"); 219 220 // send the document to the browser. The second argument to `cgi.write` 221 // indicates that this is all the data at once, enabling a few small 222 // optimizations. 223 cgi.write(document.toString(), true); 224 } 225 --- 226 227 Concepts: 228 Input: [Cgi.get], [Cgi.post], [Cgi.request], [Cgi.files], [Cgi.cookies], [Cgi.pathInfo], [Cgi.requestMethod], 229 and HTTP headers ([Cgi.headers], [Cgi.userAgent], [Cgi.referrer], [Cgi.accept], [Cgi.authorization], [Cgi.lastEventId]) 230 231 Output: [Cgi.write], [Cgi.header], [Cgi.setResponseStatus], [Cgi.setResponseContentType], [Cgi.gzipResponse] 232 233 Cookies: [Cgi.setCookie], [Cgi.clearCookie], [Cgi.cookie], [Cgi.cookies] 234 235 Caching: [Cgi.setResponseExpires], [Cgi.updateResponseExpires], [Cgi.setCache] 236 237 Redirections: [Cgi.setResponseLocation] 238 239 Other Information: [Cgi.remoteAddress], [Cgi.https], [Cgi.port], [Cgi.scriptName], [Cgi.requestUri], [Cgi.getCurrentCompleteUri], [Cgi.onRequestBodyDataReceived] 240 241 Overriding behavior: [Cgi.handleIncomingDataChunk], [Cgi.prepareForIncomingDataChunks], [Cgi.cleanUpPostDataState] 242 243 Installing: Apache, IIS, CGI, FastCGI, SCGI, embedded HTTPD (not recommended for production use) 244 245 Guide_for_PHP_users: 246 If you are coming from PHP, here's a quick guide to help you get started: 247 248 $(SIDE_BY_SIDE 249 $(COLUMN 250 ```php 251 <?php 252 $foo = $_POST["foo"]; 253 $bar = $_GET["bar"]; 254 $baz = $_COOKIE["baz"]; 255 256 $user_ip = $_SERVER["REMOTE_ADDR"]; 257 $host = $_SERVER["HTTP_HOST"]; 258 $path = $_SERVER["PATH_INFO"]; 259 260 setcookie("baz", "some value"); 261 262 echo "hello!"; 263 ?> 264 ``` 265 ) 266 $(COLUMN 267 --- 268 import arsd.cgi; 269 void app(Cgi cgi) { 270 string foo = cgi.post["foo"]; 271 string bar = cgi.get["bar"]; 272 string baz = cgi.cookies["baz"]; 273 274 string user_ip = cgi.remoteAddress; 275 string host = cgi.host; 276 string path = cgi.pathInfo; 277 278 cgi.setCookie("baz", "some value"); 279 280 cgi.write("hello!"); 281 } 282 283 mixin GenericMain!app 284 --- 285 ) 286 ) 287 288 $(H3 Array elements) 289 290 291 In PHP, you can give a form element a name like `"something[]"`, and then 292 `$_POST["something"]` gives an array. In D, you can use whatever name 293 you want, and access an array of values with the `cgi.getArray["name"]` and 294 `cgi.postArray["name"]` members. 295 296 $(H3 Databases) 297 298 PHP has a lot of stuff in its standard library. cgi.d doesn't include most 299 of these, but the rest of my arsd repository has much of it. For example, 300 to access a MySQL database, download `database.d` and `mysql.d` from my 301 github repo, and try this code (assuming, of course, your database is 302 set up): 303 304 --- 305 import arsd.cgi; 306 import arsd.mysql; 307 308 void app(Cgi cgi) { 309 auto database = new MySql("localhost", "username", "password", "database_name"); 310 foreach(row; mysql.query("SELECT count(id) FROM people")) 311 cgi.write(row[0] ~ " people in database"); 312 } 313 314 mixin GenericMain!app; 315 --- 316 317 Similar modules are available for PostgreSQL, Microsoft SQL Server, and SQLite databases, 318 implementing the same basic interface. 319 320 See_Also: 321 322 You may also want to see [arsd.dom], [arsd.web], and [arsd.html] for more code for making 323 web applications. 324 325 For working with json, try [arsd.jsvar]. 326 327 [arsd.database], [arsd.mysql], [arsd.postgres], [arsd.mssql], and [arsd.sqlite] can help in 328 accessing databases. 329 330 If you are looking to access a web application via HTTP, try [std.net.curl], [arsd.curl], or [arsd.http2]. 331 332 Copyright: 333 334 cgi.d copyright 2008-2019, Adam D. Ruppe. Provided under the Boost Software License. 335 336 Yes, this file is almost ten years old, and yes, it is still actively maintained and used. 337 +/ 338 module arsd.cgi; 339 340 static import std.file; 341 342 // for a single thread, linear request thing, use: 343 // -version=embedded_httpd_threads -version=cgi_no_threads 344 345 version(Posix) { 346 version(CRuntime_Musl) { 347 348 } else version(minimal) { 349 350 } else { 351 version(GNU) { 352 // GDC doesn't support static foreach so I had to cheat on it :( 353 } else { 354 version=with_breaking_cgi_features; 355 version=with_sendfd; 356 version=with_addon_servers; 357 } 358 } 359 } 360 361 // the servers must know about the connections to talk to them; the interfaces are vital 362 version(with_addon_servers) 363 version=with_addon_servers_connections; 364 365 version(embedded_httpd) { 366 version(linux) 367 version=embedded_httpd_processes; 368 else { 369 version=embedded_httpd_threads; 370 } 371 372 /* 373 version(with_openssl) { 374 pragma(lib, "crypto"); 375 pragma(lib, "ssl"); 376 } 377 */ 378 } 379 380 version(embedded_httpd_processes) 381 version=embedded_httpd_processes_accept_after_fork; // I am getting much better average performance on this, so just keeping it. But the other way MIGHT help keep the variation down so i wanna keep the code to play with later 382 383 version(embedded_httpd_threads) { 384 // unless the user overrides the default.. 385 version(cgi_session_server_process) 386 {} 387 else 388 version=cgi_embedded_sessions; 389 } 390 version(scgi) { 391 // unless the user overrides the default.. 392 version(cgi_session_server_process) 393 {} 394 else 395 version=cgi_embedded_sessions; 396 } 397 398 // fall back if the other is not defined so we can cleanly version it below 399 version(cgi_embedded_sessions) {} 400 else version=cgi_session_server_process; 401 402 403 version=cgi_with_websocket; 404 405 enum long defaultMaxContentLength = 5_000_000; 406 407 /* 408 409 To do a file download offer in the browser: 410 411 cgi.setResponseContentType("text/csv"); 412 cgi.header("Content-Disposition: attachment; filename=\"customers.csv\""); 413 */ 414 415 // FIXME: the location header is supposed to be an absolute url I guess. 416 417 // FIXME: would be cool to flush part of a dom document before complete 418 // somehow in here and dom.d. 419 420 421 // FIXME: 100 Continue in the nph section? Probably belongs on the 422 // httpd class though. 423 424 // these are public so you can mixin GenericMain. 425 // FIXME: use a function level import instead! 426 public import std.string; 427 public import std.stdio; 428 public import std.conv; 429 import std.uri; 430 import std.uni; 431 import std.algorithm.comparison; 432 import std.algorithm.searching; 433 import std.exception; 434 import std.base64; 435 static import std.algorithm; 436 import std.datetime; 437 import std.range; 438 439 import std.process; 440 441 import std.zlib; 442 443 444 T[] consume(T)(T[] range, int count) { 445 if(count > range.length) 446 count = range.length; 447 return range[count..$]; 448 } 449 450 int locationOf(T)(T[] data, string item) { 451 const(ubyte[]) d = cast(const(ubyte[])) data; 452 const(ubyte[]) i = cast(const(ubyte[])) item; 453 454 // this is a vague sanity check to ensure we aren't getting insanely 455 // sized input that will infinite loop below. it should never happen; 456 // even huge file uploads ought to come in smaller individual pieces. 457 if(d.length > (int.max/2)) 458 throw new Exception("excessive block of input"); 459 460 for(int a = 0; a < d.length; a++) { 461 if(a + i.length > d.length) 462 return -1; 463 if(d[a..a+i.length] == i) 464 return a; 465 } 466 467 return -1; 468 } 469 470 /// If you are doing a custom cgi class, mixing this in can take care of 471 /// the required constructors for you 472 mixin template ForwardCgiConstructors() { 473 this(long maxContentLength = defaultMaxContentLength, 474 string[string] env = null, 475 const(ubyte)[] delegate() readdata = null, 476 void delegate(const(ubyte)[]) _rawDataOutput = null, 477 void delegate() _flush = null 478 ) { super(maxContentLength, env, readdata, _rawDataOutput, _flush); } 479 480 this(string[] args) { super(args); } 481 482 this( 483 BufferedInputRange inputData, 484 string address, ushort _port, 485 int pathInfoStarts = 0, 486 bool _https = false, 487 void delegate(const(ubyte)[]) _rawDataOutput = null, 488 void delegate() _flush = null, 489 // this pointer tells if the connection is supposed to be closed after we handle this 490 bool* closeConnection = null) 491 { 492 super(inputData, address, _port, pathInfoStarts, _https, _rawDataOutput, _flush, closeConnection); 493 } 494 495 this(BufferedInputRange ir, bool* closeConnection) { super(ir, closeConnection); } 496 } 497 498 499 500 version(Windows) { 501 // FIXME: ugly hack to solve stdin exception problems on Windows: 502 // reading stdin results in StdioException (Bad file descriptor) 503 // this is probably due to http://d.puremagic.com/issues/show_bug.cgi?id=3425 504 private struct stdin { 505 struct ByChunk { // Replicates std.stdio.ByChunk 506 private: 507 ubyte[] chunk_; 508 public: 509 this(size_t size) 510 in { 511 assert(size, "size must be larger than 0"); 512 } 513 body { 514 chunk_ = new ubyte[](size); 515 popFront(); 516 } 517 518 @property bool empty() const { 519 return !std.stdio.stdin.isOpen || std.stdio.stdin.eof; // Ugly, but seems to do the job 520 } 521 @property nothrow ubyte[] front() { return chunk_; } 522 void popFront() { 523 enforce(!empty, "Cannot call popFront on empty range"); 524 chunk_ = stdin.rawRead(chunk_); 525 } 526 } 527 528 import core.sys.windows.windows; 529 static: 530 531 static this() { 532 // Set stdin to binary mode 533 version(Win64) 534 _setmode(std.stdio.stdin.fileno(), 0x8000); 535 else 536 setmode(std.stdio.stdin.fileno(), 0x8000); 537 } 538 539 T[] rawRead(T)(T[] buf) { 540 uint bytesRead; 541 auto result = ReadFile(GetStdHandle(STD_INPUT_HANDLE), buf.ptr, cast(int) (buf.length * T.sizeof), &bytesRead, null); 542 543 if (!result) { 544 auto err = GetLastError(); 545 if (err == 38/*ERROR_HANDLE_EOF*/ || err == 109/*ERROR_BROKEN_PIPE*/) // 'good' errors meaning end of input 546 return buf[0..0]; 547 // Some other error, throw it 548 549 char* buffer; 550 scope(exit) LocalFree(buffer); 551 552 // FORMAT_MESSAGE_ALLOCATE_BUFFER = 0x00000100 553 // FORMAT_MESSAGE_FROM_SYSTEM = 0x00001000 554 FormatMessageA(0x1100, null, err, 0, cast(char*)&buffer, 256, null); 555 throw new Exception(to!string(buffer)); 556 } 557 enforce(!(bytesRead % T.sizeof), "I/O error"); 558 return buf[0..bytesRead / T.sizeof]; 559 } 560 561 auto byChunk(size_t sz) { return ByChunk(sz); } 562 } 563 } 564 565 /// The main interface with the web request 566 class Cgi { 567 public: 568 /// the methods a request can be 569 enum RequestMethod { GET, HEAD, POST, PUT, DELETE, // GET and POST are the ones that really work 570 // these are defined in the standard, but idk if they are useful for anything 571 OPTIONS, TRACE, CONNECT, 572 // These seem new, I have only recently seen them 573 PATCH, MERGE, 574 // this is an extension for when the method is not specified and you want to assume 575 CommandLine } 576 577 578 /+ 579 /++ 580 Cgi provides a per-request memory pool 581 582 +/ 583 void[] allocateMemory(size_t nBytes) { 584 585 } 586 587 /// ditto 588 void[] reallocateMemory(void[] old, size_t nBytes) { 589 590 } 591 592 /// ditto 593 void freeMemory(void[] memory) { 594 595 } 596 +/ 597 598 599 /* 600 import core.runtime; 601 auto args = Runtime.args(); 602 603 we can call the app a few ways: 604 605 1) set up the environment variables and call the app (manually simulating CGI) 606 2) simulate a call automatically: 607 ./app method 'uri' 608 609 for example: 610 ./app get /path?arg arg2=something 611 612 Anything on the uri is treated as query string etc 613 614 on get method, further args are appended to the query string (encoded automatically) 615 on post method, further args are done as post 616 617 618 @name means import from file "name". if name == -, it uses stdin 619 (so info=@- means set info to the value of stdin) 620 621 622 Other arguments include: 623 --cookie name=value (these are all concated together) 624 --header 'X-Something: cool' 625 --referrer 'something' 626 --port 80 627 --remote-address some.ip.address.here 628 --https yes 629 --user-agent 'something' 630 --userpass 'user:pass' 631 --authorization 'Basic base64encoded_user:pass' 632 --accept 'content' // FIXME: better example 633 --last-event-id 'something' 634 --host 'something.com' 635 636 Non-simulation arguments: 637 --port xxx listening port for non-cgi things (valid for the cgi interfaces) 638 --listening-host the ip address the application should listen on, or if you want to use unix domain sockets, it is here you can set them: `--listening-host unix:filename` or, on Linux, `--listening-host abstract:name`. 639 640 */ 641 642 /** Initializes it with command line arguments (for easy testing) */ 643 this(string[] args, void delegate(const(ubyte)[]) _rawDataOutput = null) { 644 rawDataOutput = _rawDataOutput; 645 // these are all set locally so the loop works 646 // without triggering errors in dmd 2.064 647 // we go ahead and set them at the end of it to the this version 648 int port; 649 string referrer; 650 string remoteAddress; 651 string userAgent; 652 string authorization; 653 string origin; 654 string accept; 655 string lastEventId; 656 bool https; 657 string host; 658 RequestMethod requestMethod; 659 string requestUri; 660 string pathInfo; 661 string queryString; 662 663 bool lookingForMethod; 664 bool lookingForUri; 665 string nextArgIs; 666 667 string _cookie; 668 string _queryString; 669 string[][string] _post; 670 string[string] _headers; 671 672 string[] breakUp(string s) { 673 string k, v; 674 auto idx = s.indexOf("="); 675 if(idx == -1) { 676 k = s; 677 } else { 678 k = s[0 .. idx]; 679 v = s[idx + 1 .. $]; 680 } 681 682 return [k, v]; 683 } 684 685 lookingForMethod = true; 686 687 scriptName = args[0]; 688 scriptFileName = args[0]; 689 690 environmentVariables = cast(const) environment.toAA; 691 692 foreach(arg; args[1 .. $]) { 693 if(arg.startsWith("--")) { 694 nextArgIs = arg[2 .. $]; 695 } else if(nextArgIs.length) { 696 if (nextArgIs == "cookie") { 697 auto info = breakUp(arg); 698 if(_cookie.length) 699 _cookie ~= "; "; 700 _cookie ~= std.uri.encodeComponent(info[0]) ~ "=" ~ std.uri.encodeComponent(info[1]); 701 } 702 else if (nextArgIs == "port") { 703 port = to!int(arg); 704 } 705 else if (nextArgIs == "referrer") { 706 referrer = arg; 707 } 708 else if (nextArgIs == "remote-address") { 709 remoteAddress = arg; 710 } 711 else if (nextArgIs == "user-agent") { 712 userAgent = arg; 713 } 714 else if (nextArgIs == "authorization") { 715 authorization = arg; 716 } 717 else if (nextArgIs == "userpass") { 718 authorization = "Basic " ~ Base64.encode(cast(immutable(ubyte)[]) (arg)).idup; 719 } 720 else if (nextArgIs == "origin") { 721 origin = arg; 722 } 723 else if (nextArgIs == "accept") { 724 accept = arg; 725 } 726 else if (nextArgIs == "last-event-id") { 727 lastEventId = arg; 728 } 729 else if (nextArgIs == "https") { 730 if(arg == "yes") 731 https = true; 732 } 733 else if (nextArgIs == "header") { 734 string thing, other; 735 auto idx = arg.indexOf(":"); 736 if(idx == -1) 737 throw new Exception("need a colon in a http header"); 738 thing = arg[0 .. idx]; 739 other = arg[idx + 1.. $]; 740 _headers[thing.strip.toLower()] = other.strip; 741 } 742 else if (nextArgIs == "host") { 743 host = arg; 744 } 745 // else 746 // skip, we don't know it but that's ok, it might be used elsewhere so no error 747 748 nextArgIs = null; 749 } else if(lookingForMethod) { 750 lookingForMethod = false; 751 lookingForUri = true; 752 753 if(arg.asLowerCase().equal("commandline")) 754 requestMethod = RequestMethod.CommandLine; 755 else 756 requestMethod = to!RequestMethod(arg.toUpper()); 757 } else if(lookingForUri) { 758 lookingForUri = false; 759 760 requestUri = arg; 761 762 auto idx = arg.indexOf("?"); 763 if(idx == -1) 764 pathInfo = arg; 765 else { 766 pathInfo = arg[0 .. idx]; 767 _queryString = arg[idx + 1 .. $]; 768 } 769 } else { 770 // it is an argument of some sort 771 if(requestMethod == Cgi.RequestMethod.POST || requestMethod == Cgi.RequestMethod.PATCH || requestMethod == Cgi.RequestMethod.PUT) { 772 auto parts = breakUp(arg); 773 _post[parts[0]] ~= parts[1]; 774 allPostNamesInOrder ~= parts[0]; 775 allPostValuesInOrder ~= parts[1]; 776 } else { 777 if(_queryString.length) 778 _queryString ~= "&"; 779 auto parts = breakUp(arg); 780 _queryString ~= std.uri.encodeComponent(parts[0]) ~ "=" ~ std.uri.encodeComponent(parts[1]); 781 } 782 } 783 } 784 785 acceptsGzip = false; 786 keepAliveRequested = false; 787 requestHeaders = cast(immutable) _headers; 788 789 cookie = _cookie; 790 cookiesArray = getCookieArray(); 791 cookies = keepLastOf(cookiesArray); 792 793 queryString = _queryString; 794 getArray = cast(immutable) decodeVariables(queryString, "&", &allGetNamesInOrder, &allGetValuesInOrder); 795 get = keepLastOf(getArray); 796 797 postArray = cast(immutable) _post; 798 post = keepLastOf(_post); 799 800 // FIXME 801 filesArray = null; 802 files = null; 803 804 isCalledWithCommandLineArguments = true; 805 806 this.port = port; 807 this.referrer = referrer; 808 this.remoteAddress = remoteAddress; 809 this.userAgent = userAgent; 810 this.authorization = authorization; 811 this.origin = origin; 812 this.accept = accept; 813 this.lastEventId = lastEventId; 814 this.https = https; 815 this.host = host; 816 this.requestMethod = requestMethod; 817 this.requestUri = requestUri; 818 this.pathInfo = pathInfo; 819 this.queryString = queryString; 820 this.postJson = null; 821 } 822 823 private { 824 string[] allPostNamesInOrder; 825 string[] allPostValuesInOrder; 826 string[] allGetNamesInOrder; 827 string[] allGetValuesInOrder; 828 } 829 830 CgiConnectionHandle getOutputFileHandle() { 831 return _outputFileHandle; 832 } 833 834 CgiConnectionHandle _outputFileHandle = INVALID_CGI_CONNECTION_HANDLE; 835 836 /** Initializes it using a CGI or CGI-like interface */ 837 this(long maxContentLength = defaultMaxContentLength, 838 // use this to override the environment variable listing 839 in string[string] env = null, 840 // and this should return a chunk of data. return empty when done 841 const(ubyte)[] delegate() readdata = null, 842 // finally, use this to do custom output if needed 843 void delegate(const(ubyte)[]) _rawDataOutput = null, 844 // to flush teh custom output 845 void delegate() _flush = null 846 ) 847 { 848 849 // these are all set locally so the loop works 850 // without triggering errors in dmd 2.064 851 // we go ahead and set them at the end of it to the this version 852 int port; 853 string referrer; 854 string remoteAddress; 855 string userAgent; 856 string authorization; 857 string origin; 858 string accept; 859 string lastEventId; 860 bool https; 861 string host; 862 RequestMethod requestMethod; 863 string requestUri; 864 string pathInfo; 865 string queryString; 866 867 868 869 isCalledWithCommandLineArguments = false; 870 rawDataOutput = _rawDataOutput; 871 flushDelegate = _flush; 872 auto getenv = delegate string(string var) { 873 if(env is null) 874 return std.process.environment.get(var); 875 auto e = var in env; 876 if(e is null) 877 return null; 878 return *e; 879 }; 880 881 environmentVariables = env is null ? 882 cast(const) environment.toAA : 883 env; 884 885 // fetching all the request headers 886 string[string] requestHeadersHere; 887 foreach(k, v; env is null ? cast(const) environment.toAA() : env) { 888 if(k.startsWith("HTTP_")) { 889 requestHeadersHere[replace(k["HTTP_".length .. $].toLower(), "_", "-")] = v; 890 } 891 } 892 893 this.requestHeaders = assumeUnique(requestHeadersHere); 894 895 requestUri = getenv("REQUEST_URI"); 896 897 cookie = getenv("HTTP_COOKIE"); 898 cookiesArray = getCookieArray(); 899 cookies = keepLastOf(cookiesArray); 900 901 referrer = getenv("HTTP_REFERER"); 902 userAgent = getenv("HTTP_USER_AGENT"); 903 remoteAddress = getenv("REMOTE_ADDR"); 904 host = getenv("HTTP_HOST"); 905 pathInfo = getenv("PATH_INFO"); 906 907 queryString = getenv("QUERY_STRING"); 908 scriptName = getenv("SCRIPT_NAME"); 909 { 910 import core.runtime; 911 auto sfn = getenv("SCRIPT_FILENAME"); 912 scriptFileName = sfn.length ? sfn : Runtime.args[0]; 913 } 914 915 bool iis = false; 916 917 // Because IIS doesn't pass requestUri, we simulate it here if it's empty. 918 if(requestUri.length == 0) { 919 // IIS sometimes includes the script name as part of the path info - we don't want that 920 if(pathInfo.length >= scriptName.length && (pathInfo[0 .. scriptName.length] == scriptName)) 921 pathInfo = pathInfo[scriptName.length .. $]; 922 923 requestUri = scriptName ~ pathInfo ~ (queryString.length ? ("?" ~ queryString) : ""); 924 925 iis = true; // FIXME HACK - used in byChunk below - see bugzilla 6339 926 927 // FIXME: this works for apache and iis... but what about others? 928 } 929 930 931 auto ugh = decodeVariables(queryString, "&", &allGetNamesInOrder, &allGetValuesInOrder); 932 getArray = assumeUnique(ugh); 933 get = keepLastOf(getArray); 934 935 936 // NOTE: on shitpache, you need to specifically forward this 937 authorization = getenv("HTTP_AUTHORIZATION"); 938 // this is a hack because Apache is a shitload of fuck and 939 // refuses to send the real header to us. Compatible 940 // programs should send both the standard and X- versions 941 942 // NOTE: if you have access to .htaccess or httpd.conf, you can make this 943 // unnecessary with mod_rewrite, so it is commented 944 945 //if(authorization.length == 0) // if the std is there, use it 946 // authorization = getenv("HTTP_X_AUTHORIZATION"); 947 948 // the REDIRECT_HTTPS check is here because with an Apache hack, the port can become wrong 949 if(getenv("SERVER_PORT").length && getenv("REDIRECT_HTTPS") != "on") 950 port = to!int(getenv("SERVER_PORT")); 951 else 952 port = 0; // this was probably called from the command line 953 954 auto ae = getenv("HTTP_ACCEPT_ENCODING"); 955 if(ae.length && ae.indexOf("gzip") != -1) 956 acceptsGzip = true; 957 958 accept = getenv("HTTP_ACCEPT"); 959 lastEventId = getenv("HTTP_LAST_EVENT_ID"); 960 961 auto ka = getenv("HTTP_CONNECTION"); 962 if(ka.length && ka.asLowerCase().canFind("keep-alive")) 963 keepAliveRequested = true; 964 965 auto or = getenv("HTTP_ORIGIN"); 966 origin = or; 967 968 auto rm = getenv("REQUEST_METHOD"); 969 if(rm.length) 970 requestMethod = to!RequestMethod(getenv("REQUEST_METHOD")); 971 else 972 requestMethod = RequestMethod.CommandLine; 973 974 // FIXME: hack on REDIRECT_HTTPS; this is there because the work app uses mod_rewrite which loses the https flag! So I set it with [E=HTTPS=%HTTPS] or whatever but then it gets translated to here so i want it to still work. This is arguably wrong but meh. 975 https = (getenv("HTTPS") == "on" || getenv("REDIRECT_HTTPS") == "on"); 976 977 // FIXME: DOCUMENT_ROOT? 978 979 // FIXME: what about PUT? 980 if(requestMethod == RequestMethod.POST || requestMethod == Cgi.RequestMethod.PATCH || requestMethod == Cgi.RequestMethod.PUT) { 981 version(preserveData) // a hack to make forwarding simpler 982 immutable(ubyte)[] data; 983 size_t amountReceived = 0; 984 auto contentType = getenv("CONTENT_TYPE"); 985 986 // FIXME: is this ever not going to be set? I guess it depends 987 // on if the server de-chunks and buffers... seems like it has potential 988 // to be slow if they did that. The spec says it is always there though. 989 // And it has worked reliably for me all year in the live environment, 990 // but some servers might be different. 991 auto contentLength = to!size_t(getenv("CONTENT_LENGTH")); 992 993 immutable originalContentLength = contentLength; 994 if(contentLength) { 995 if(maxContentLength > 0 && contentLength > maxContentLength) { 996 setResponseStatus("413 Request entity too large"); 997 write("You tried to upload a file that is too large."); 998 close(); 999 throw new Exception("POST too large"); 1000 } 1001 prepareForIncomingDataChunks(contentType, contentLength); 1002 1003 1004 int processChunk(in ubyte[] chunk) { 1005 if(chunk.length > contentLength) { 1006 handleIncomingDataChunk(chunk[0..contentLength]); 1007 amountReceived += contentLength; 1008 contentLength = 0; 1009 return 1; 1010 } else { 1011 handleIncomingDataChunk(chunk); 1012 contentLength -= chunk.length; 1013 amountReceived += chunk.length; 1014 } 1015 if(contentLength == 0) 1016 return 1; 1017 1018 onRequestBodyDataReceived(amountReceived, originalContentLength); 1019 return 0; 1020 } 1021 1022 1023 if(readdata is null) { 1024 foreach(ubyte[] chunk; stdin.byChunk(iis ? contentLength : 4096)) 1025 if(processChunk(chunk)) 1026 break; 1027 } else { 1028 // we have a custom data source.. 1029 auto chunk = readdata(); 1030 while(chunk.length) { 1031 if(processChunk(chunk)) 1032 break; 1033 chunk = readdata(); 1034 } 1035 } 1036 1037 onRequestBodyDataReceived(amountReceived, originalContentLength); 1038 postArray = assumeUnique(pps._post); 1039 filesArray = assumeUnique(pps._files); 1040 files = keepLastOf(filesArray); 1041 post = keepLastOf(postArray); 1042 this.postJson = pps.postJson; 1043 cleanUpPostDataState(); 1044 } 1045 1046 version(preserveData) 1047 originalPostData = data; 1048 } 1049 // fixme: remote_user script name 1050 1051 1052 this.port = port; 1053 this.referrer = referrer; 1054 this.remoteAddress = remoteAddress; 1055 this.userAgent = userAgent; 1056 this.authorization = authorization; 1057 this.origin = origin; 1058 this.accept = accept; 1059 this.lastEventId = lastEventId; 1060 this.https = https; 1061 this.host = host; 1062 this.requestMethod = requestMethod; 1063 this.requestUri = requestUri; 1064 this.pathInfo = pathInfo; 1065 this.queryString = queryString; 1066 } 1067 1068 /// Cleans up any temporary files. Do not use the object 1069 /// after calling this. 1070 /// 1071 /// NOTE: it is called automatically by GenericMain 1072 // FIXME: this should be called if the constructor fails too, if it has created some garbage... 1073 void dispose() { 1074 foreach(file; files) { 1075 if(!file.contentInMemory) 1076 if(std.file.exists(file.contentFilename)) 1077 std.file.remove(file.contentFilename); 1078 } 1079 } 1080 1081 private { 1082 struct PostParserState { 1083 string contentType; 1084 string boundary; 1085 string localBoundary; // the ones used at the end or something lol 1086 bool isMultipart; 1087 bool isJson; 1088 1089 ulong expectedLength; 1090 ulong contentConsumed; 1091 immutable(ubyte)[] buffer; 1092 1093 // multipart parsing state 1094 int whatDoWeWant; 1095 bool weHaveAPart; 1096 string[] thisOnesHeaders; 1097 immutable(ubyte)[] thisOnesData; 1098 1099 string postJson; 1100 1101 UploadedFile piece; 1102 bool isFile = false; 1103 1104 size_t memoryCommitted; 1105 1106 // do NOT keep mutable references to these anywhere! 1107 // I assume they are unique in the constructor once we're all done getting data. 1108 string[][string] _post; 1109 UploadedFile[][string] _files; 1110 } 1111 1112 PostParserState pps; 1113 } 1114 1115 /// This represents a file the user uploaded via a POST request. 1116 static struct UploadedFile { 1117 /// If you want to create one of these structs for yourself from some data, 1118 /// use this function. 1119 static UploadedFile fromData(immutable(void)[] data, string name = null) { 1120 Cgi.UploadedFile f; 1121 f.filename = name; 1122 f.content = cast(immutable(ubyte)[]) data; 1123 f.contentInMemory = true; 1124 return f; 1125 } 1126 1127 string name; /// The name of the form element. 1128 string filename; /// The filename the user set. 1129 string contentType; /// The MIME type the user's browser reported. (Not reliable.) 1130 1131 /** 1132 For small files, cgi.d will buffer the uploaded file in memory, and make it 1133 directly accessible to you through the content member. I find this very convenient 1134 and somewhat efficient, since it can avoid hitting the disk entirely. (I 1135 often want to inspect and modify the file anyway!) 1136 1137 I find the file is very large, it is undesirable to eat that much memory just 1138 for a file buffer. In those cases, if you pass a large enough value for maxContentLength 1139 to the constructor so they are accepted, cgi.d will write the content to a temporary 1140 file that you can re-read later. 1141 1142 You can override this behavior by subclassing Cgi and overriding the protected 1143 handlePostChunk method. Note that the object is not initialized when you 1144 write that method - the http headers are available, but the cgi.post method 1145 is not. You may parse the file as it streams in using this method. 1146 1147 1148 Anyway, if the file is small enough to be in memory, contentInMemory will be 1149 set to true, and the content is available in the content member. 1150 1151 If not, contentInMemory will be set to false, and the content saved in a file, 1152 whose name will be available in the contentFilename member. 1153 1154 1155 Tip: if you know you are always dealing with small files, and want the convenience 1156 of ignoring this member, construct Cgi with a small maxContentLength. Then, if 1157 a large file comes in, it simply throws an exception (and HTTP error response) 1158 instead of trying to handle it. 1159 1160 The default value of maxContentLength in the constructor is for small files. 1161 */ 1162 bool contentInMemory = true; // the default ought to always be true 1163 immutable(ubyte)[] content; /// The actual content of the file, if contentInMemory == true 1164 string contentFilename; /// the file where we dumped the content, if contentInMemory == false. Note that if you want to keep it, you MUST move the file, since otherwise it is considered garbage when cgi is disposed. 1165 1166 /// 1167 ulong fileSize() { 1168 if(contentInMemory) 1169 return content.length; 1170 import std.file; 1171 return std.file.getSize(contentFilename); 1172 1173 } 1174 1175 /// 1176 void writeToFile(string filenameToSaveTo) const { 1177 import std.file; 1178 if(contentInMemory) 1179 std.file.write(filenameToSaveTo, content); 1180 else 1181 std.file.rename(contentFilename, filenameToSaveTo); 1182 } 1183 } 1184 1185 // given a content type and length, decide what we're going to do with the data.. 1186 protected void prepareForIncomingDataChunks(string contentType, ulong contentLength) { 1187 pps.expectedLength = contentLength; 1188 1189 auto terminator = contentType.indexOf(";"); 1190 if(terminator == -1) 1191 terminator = contentType.length; 1192 1193 pps.contentType = contentType[0 .. terminator]; 1194 auto b = contentType[terminator .. $]; 1195 if(b.length) { 1196 auto idx = b.indexOf("boundary="); 1197 if(idx != -1) { 1198 pps.boundary = b[idx + "boundary=".length .. $]; 1199 pps.localBoundary = "\r\n--" ~ pps.boundary; 1200 } 1201 } 1202 1203 // while a content type SHOULD be sent according to the RFC, it is 1204 // not required. We're told we SHOULD guess by looking at the content 1205 // but it seems to me that this only happens when it is urlencoded. 1206 if(pps.contentType == "application/x-www-form-urlencoded" || pps.contentType == "") { 1207 pps.isMultipart = false; 1208 } else if(pps.contentType == "multipart/form-data") { 1209 pps.isMultipart = true; 1210 enforce(pps.boundary.length, "no boundary"); 1211 } else if(pps.contentType == "text/plain") { 1212 pps.isMultipart = false; 1213 pps.isJson = true; // FIXME: hack, it isn't actually this 1214 } else if(pps.contentType == "text/xml") { // FIXME: what if we used this as a fallback? 1215 pps.isMultipart = false; 1216 pps.isJson = true; // FIXME: hack, it isn't actually this 1217 } else if(pps.contentType == "application/json") { 1218 pps.isJson = true; 1219 pps.isMultipart = false; 1220 //} else if(pps.contentType == "application/json") { 1221 //pps.isJson = true; 1222 } else { 1223 // FIXME: should set a http error code too 1224 throw new Exception("unknown request content type: " ~ pps.contentType); 1225 } 1226 } 1227 1228 // handles streaming POST data. If you handle some other content type, you should 1229 // override this. If the data isn't the content type you want, you ought to call 1230 // super.handleIncomingDataChunk so regular forms and files still work. 1231 1232 // FIXME: I do some copying in here that I'm pretty sure is unnecessary, and the 1233 // file stuff I'm sure is inefficient. But, my guess is the real bottleneck is network 1234 // input anyway, so I'm not going to get too worked up about it right now. 1235 protected void handleIncomingDataChunk(const(ubyte)[] chunk) { 1236 if(chunk.length == 0) 1237 return; 1238 assert(chunk.length <= 32 * 1024 * 1024); // we use chunk size as a memory constraint thing, so 1239 // if we're passed big chunks, it might throw unnecessarily. 1240 // just pass it smaller chunks at a time. 1241 if(pps.isMultipart) { 1242 // multipart/form-data 1243 1244 1245 void pieceHasNewContent() { 1246 // we just grew the piece's buffer. Do we have to switch to file backing? 1247 if(pps.piece.contentInMemory) { 1248 if(pps.piece.content.length <= 10 * 1024 * 1024) 1249 // meh, I'm ok with it. 1250 return; 1251 else { 1252 // this is too big. 1253 if(!pps.isFile) 1254 throw new Exception("Request entity too large"); // a variable this big is kinda ridiculous, just reject it. 1255 else { 1256 // a file this large is probably acceptable though... let's use a backing file. 1257 pps.piece.contentInMemory = false; 1258 // FIXME: say... how do we intend to delete these things? cgi.dispose perhaps. 1259 1260 int count = 0; 1261 pps.piece.contentFilename = getTempDirectory() ~ "arsd_cgi_uploaded_file_" ~ to!string(getUtcTime()) ~ "-" ~ to!string(count); 1262 // odds are this loop will never be entered, but we want it just in case. 1263 while(std.file.exists(pps.piece.contentFilename)) { 1264 count++; 1265 pps.piece.contentFilename = getTempDirectory() ~ "arsd_cgi_uploaded_file_" ~ to!string(getUtcTime()) ~ "-" ~ to!string(count); 1266 } 1267 // I hope this creates the file pretty quickly, or the loop might be useless... 1268 // FIXME: maybe I should write some kind of custom transaction here. 1269 std.file.write(pps.piece.contentFilename, pps.piece.content); 1270 1271 pps.piece.content = null; 1272 } 1273 } 1274 } else { 1275 // it's already in a file, so just append it to what we have 1276 if(pps.piece.content.length) { 1277 // FIXME: this is surely very inefficient... we'll be calling this by 4kb chunk... 1278 std.file.append(pps.piece.contentFilename, pps.piece.content); 1279 pps.piece.content = null; 1280 } 1281 } 1282 } 1283 1284 1285 void commitPart() { 1286 if(!pps.weHaveAPart) 1287 return; 1288 1289 pieceHasNewContent(); // be sure the new content is handled every time 1290 1291 if(pps.isFile) { 1292 // I'm not sure if other environments put files in post or not... 1293 // I used to not do it, but I think I should, since it is there... 1294 pps._post[pps.piece.name] ~= pps.piece.filename; 1295 pps._files[pps.piece.name] ~= pps.piece; 1296 1297 allPostNamesInOrder ~= pps.piece.name; 1298 allPostValuesInOrder ~= pps.piece.filename; 1299 } else { 1300 pps._post[pps.piece.name] ~= cast(string) pps.piece.content; 1301 1302 allPostNamesInOrder ~= pps.piece.name; 1303 allPostValuesInOrder ~= cast(string) pps.piece.content; 1304 } 1305 1306 /* 1307 stderr.writeln("RECEIVED: ", pps.piece.name, "=", 1308 pps.piece.content.length < 1000 1309 ? 1310 to!string(pps.piece.content) 1311 : 1312 "too long"); 1313 */ 1314 1315 // FIXME: the limit here 1316 pps.memoryCommitted += pps.piece.content.length; 1317 1318 pps.weHaveAPart = false; 1319 pps.whatDoWeWant = 1; 1320 pps.thisOnesHeaders = null; 1321 pps.thisOnesData = null; 1322 1323 pps.piece = UploadedFile.init; 1324 pps.isFile = false; 1325 } 1326 1327 void acceptChunk() { 1328 pps.buffer ~= chunk; 1329 chunk = null; // we've consumed it into the buffer, so keeping it just brings confusion 1330 } 1331 1332 immutable(ubyte)[] consume(size_t howMuch) { 1333 pps.contentConsumed += howMuch; 1334 auto ret = pps.buffer[0 .. howMuch]; 1335 pps.buffer = pps.buffer[howMuch .. $]; 1336 return ret; 1337 } 1338 1339 dataConsumptionLoop: do { 1340 switch(pps.whatDoWeWant) { 1341 default: assert(0); 1342 case 0: 1343 acceptChunk(); 1344 // the format begins with two extra leading dashes, then we should be at the boundary 1345 if(pps.buffer.length < 2) 1346 return; 1347 assert(pps.buffer[0] == '-', "no leading dash"); 1348 consume(1); 1349 assert(pps.buffer[0] == '-', "no second leading dash"); 1350 consume(1); 1351 1352 pps.whatDoWeWant = 1; 1353 goto case 1; 1354 /* fallthrough */ 1355 case 1: // looking for headers 1356 // here, we should be lined up right at the boundary, which is followed by a \r\n 1357 1358 // want to keep the buffer under control in case we're under attack 1359 //stderr.writeln("here once"); 1360 //if(pps.buffer.length + chunk.length > 70 * 1024) // they should be < 1 kb really.... 1361 // throw new Exception("wtf is up with the huge mime part headers"); 1362 1363 acceptChunk(); 1364 1365 if(pps.buffer.length < pps.boundary.length) 1366 return; // not enough data, since there should always be a boundary here at least 1367 1368 if(pps.contentConsumed + pps.boundary.length + 6 == pps.expectedLength) { 1369 assert(pps.buffer.length == pps.boundary.length + 4 + 2); // --, --, and \r\n 1370 // we *should* be at the end here! 1371 assert(pps.buffer[0] == '-'); 1372 consume(1); 1373 assert(pps.buffer[0] == '-'); 1374 consume(1); 1375 1376 // the message is terminated by --BOUNDARY--\r\n (after a \r\n leading to the boundary) 1377 assert(pps.buffer[0 .. pps.boundary.length] == cast(const(ubyte[])) pps.boundary, 1378 "not lined up on boundary " ~ pps.boundary); 1379 consume(pps.boundary.length); 1380 1381 assert(pps.buffer[0] == '-'); 1382 consume(1); 1383 assert(pps.buffer[0] == '-'); 1384 consume(1); 1385 1386 assert(pps.buffer[0] == '\r'); 1387 consume(1); 1388 assert(pps.buffer[0] == '\n'); 1389 consume(1); 1390 1391 assert(pps.buffer.length == 0); 1392 assert(pps.contentConsumed == pps.expectedLength); 1393 break dataConsumptionLoop; // we're done! 1394 } else { 1395 // we're not done yet. We should be lined up on a boundary. 1396 1397 // But, we want to ensure the headers are here before we consume anything! 1398 auto headerEndLocation = locationOf(pps.buffer, "\r\n\r\n"); 1399 if(headerEndLocation == -1) 1400 return; // they *should* all be here, so we can handle them all at once. 1401 1402 assert(pps.buffer[0 .. pps.boundary.length] == cast(const(ubyte[])) pps.boundary, 1403 "not lined up on boundary " ~ pps.boundary); 1404 1405 consume(pps.boundary.length); 1406 // the boundary is always followed by a \r\n 1407 assert(pps.buffer[0] == '\r'); 1408 consume(1); 1409 assert(pps.buffer[0] == '\n'); 1410 consume(1); 1411 } 1412 1413 // re-running since by consuming the boundary, we invalidate the old index. 1414 auto headerEndLocation = locationOf(pps.buffer, "\r\n\r\n"); 1415 assert(headerEndLocation >= 0, "no header"); 1416 auto thisOnesHeaders = pps.buffer[0..headerEndLocation]; 1417 1418 consume(headerEndLocation + 4); // The +4 is the \r\n\r\n that caps it off 1419 1420 pps.thisOnesHeaders = split(cast(string) thisOnesHeaders, "\r\n"); 1421 1422 // now we'll parse the headers 1423 foreach(h; pps.thisOnesHeaders) { 1424 auto p = h.indexOf(":"); 1425 assert(p != -1, "no colon in header, got " ~ to!string(pps.thisOnesHeaders)); 1426 string hn = h[0..p]; 1427 string hv = h[p+2..$]; 1428 1429 switch(hn.toLower) { 1430 default: assert(0); 1431 case "content-disposition": 1432 auto info = hv.split("; "); 1433 foreach(i; info[1..$]) { // skipping the form-data 1434 auto o = i.split("="); // FIXME 1435 string pn = o[0]; 1436 string pv = o[1][1..$-1]; 1437 1438 if(pn == "name") { 1439 pps.piece.name = pv; 1440 } else if (pn == "filename") { 1441 pps.piece.filename = pv; 1442 pps.isFile = true; 1443 } 1444 } 1445 break; 1446 case "content-type": 1447 pps.piece.contentType = hv; 1448 break; 1449 } 1450 } 1451 1452 pps.whatDoWeWant++; // move to the next step - the data 1453 break; 1454 case 2: 1455 // when we get here, pps.buffer should contain our first chunk of data 1456 1457 if(pps.buffer.length + chunk.length > 8 * 1024 * 1024) // we might buffer quite a bit but not much 1458 throw new Exception("wtf is up with the huge mime part buffer"); 1459 1460 acceptChunk(); 1461 1462 // so the trick is, we want to process all the data up to the boundary, 1463 // but what if the chunk's end cuts the boundary off? If we're unsure, we 1464 // want to wait for the next chunk. We start by looking for the whole boundary 1465 // in the buffer somewhere. 1466 1467 auto boundaryLocation = locationOf(pps.buffer, pps.localBoundary); 1468 // assert(boundaryLocation != -1, "should have seen "~to!string(cast(ubyte[]) pps.localBoundary)~" in " ~ to!string(pps.buffer)); 1469 if(boundaryLocation != -1) { 1470 // this is easy - we can see it in it's entirety! 1471 1472 pps.piece.content ~= consume(boundaryLocation); 1473 1474 assert(pps.buffer[0] == '\r'); 1475 consume(1); 1476 assert(pps.buffer[0] == '\n'); 1477 consume(1); 1478 assert(pps.buffer[0] == '-'); 1479 consume(1); 1480 assert(pps.buffer[0] == '-'); 1481 consume(1); 1482 // the boundary here is always preceded by \r\n--, which is why we used localBoundary instead of boundary to locate it. Cut that off. 1483 pps.weHaveAPart = true; 1484 pps.whatDoWeWant = 1; // back to getting headers for the next part 1485 1486 commitPart(); // we're done here 1487 } else { 1488 // we can't see the whole thing, but what if there's a partial boundary? 1489 1490 enforce(pps.localBoundary.length < 128); // the boundary ought to be less than a line... 1491 assert(pps.localBoundary.length > 1); // should already be sane but just in case 1492 bool potentialBoundaryFound = false; 1493 1494 boundaryCheck: for(int a = 1; a < pps.localBoundary.length; a++) { 1495 // we grow the boundary a bit each time. If we think it looks the 1496 // same, better pull another chunk to be sure it's not the end. 1497 // Starting small because exiting the loop early is desirable, since 1498 // we're not keeping any ambiguity and 1 / 256 chance of exiting is 1499 // the best we can do. 1500 if(a > pps.buffer.length) 1501 break; // FIXME: is this right? 1502 assert(a <= pps.buffer.length); 1503 assert(a > 0); 1504 if(std.algorithm.endsWith(pps.buffer, pps.localBoundary[0 .. a])) { 1505 // ok, there *might* be a boundary here, so let's 1506 // not treat the end as data yet. The rest is good to 1507 // use though, since if there was a boundary there, we'd 1508 // have handled it up above after locationOf. 1509 1510 pps.piece.content ~= pps.buffer[0 .. $ - a]; 1511 consume(pps.buffer.length - a); 1512 pieceHasNewContent(); 1513 potentialBoundaryFound = true; 1514 break boundaryCheck; 1515 } 1516 } 1517 1518 if(!potentialBoundaryFound) { 1519 // we can consume the whole thing 1520 pps.piece.content ~= pps.buffer; 1521 pieceHasNewContent(); 1522 consume(pps.buffer.length); 1523 } else { 1524 // we found a possible boundary, but there was 1525 // insufficient data to be sure. 1526 assert(pps.buffer == cast(const(ubyte[])) pps.localBoundary[0 .. pps.buffer.length]); 1527 1528 return; // wait for the next chunk. 1529 } 1530 } 1531 } 1532 } while(pps.buffer.length); 1533 1534 // btw all boundaries except the first should have a \r\n before them 1535 } else { 1536 // application/x-www-form-urlencoded and application/json 1537 1538 // not using maxContentLength because that might be cranked up to allow 1539 // large file uploads. We can handle them, but a huge post[] isn't any good. 1540 if(pps.buffer.length + chunk.length > 8 * 1024 * 1024) // surely this is plenty big enough 1541 throw new Exception("wtf is up with such a gigantic form submission????"); 1542 1543 pps.buffer ~= chunk; 1544 1545 // simple handling, but it works... until someone bombs us with gigabytes of crap at least... 1546 if(pps.buffer.length == pps.expectedLength) { 1547 if(pps.isJson) 1548 pps.postJson = cast(string) pps.buffer; 1549 else 1550 pps._post = decodeVariables(cast(string) pps.buffer, "&", &allPostNamesInOrder, &allPostValuesInOrder); 1551 version(preserveData) 1552 originalPostData = pps.buffer; 1553 } else { 1554 // just for debugging 1555 } 1556 } 1557 } 1558 1559 protected void cleanUpPostDataState() { 1560 pps = PostParserState.init; 1561 } 1562 1563 /// you can override this function to somehow react 1564 /// to an upload in progress. 1565 /// 1566 /// Take note that parts of the CGI object is not yet 1567 /// initialized! Stuff from HTTP headers, including get[], is usable. 1568 /// But, none of post[] is usable, and you cannot write here. That's 1569 /// why this method is const - mutating the object won't do much anyway. 1570 /// 1571 /// My idea here was so you can output a progress bar or 1572 /// something to a cooperative client (see arsd.rtud for a potential helper) 1573 /// 1574 /// The default is to do nothing. Subclass cgi and use the 1575 /// CustomCgiMain mixin to do something here. 1576 void onRequestBodyDataReceived(size_t receivedSoFar, size_t totalExpected) const { 1577 // This space intentionally left blank. 1578 } 1579 1580 /// Initializes the cgi from completely raw HTTP data. The ir must have a Socket source. 1581 /// *closeConnection will be set to true if you should close the connection after handling this request 1582 this(BufferedInputRange ir, bool* closeConnection) { 1583 isCalledWithCommandLineArguments = false; 1584 import al = std.algorithm; 1585 1586 immutable(ubyte)[] data; 1587 1588 void rdo(const(ubyte)[] d) { 1589 sendAll(ir.source, d); 1590 } 1591 1592 this(ir, ir.source.remoteAddress().toString(), 80 /* FIXME */, 0, false, &rdo, null, closeConnection); 1593 } 1594 1595 /** 1596 Initializes it from raw HTTP request data. GenericMain uses this when you compile with -version=embedded_httpd. 1597 1598 NOTE: If you are behind a reverse proxy, the values here might not be what you expect.... it will use X-Forwarded-For for remote IP and X-Forwarded-Host for host 1599 1600 Params: 1601 inputData = the incoming data, including headers and other raw http data. 1602 When the constructor exits, it will leave this range exactly at the start of 1603 the next request on the connection (if there is one). 1604 1605 address = the IP address of the remote user 1606 _port = the port number of the connection 1607 pathInfoStarts = the offset into the path component of the http header where the SCRIPT_NAME ends and the PATH_INFO begins. 1608 _https = if this connection is encrypted (note that the input data must not actually be encrypted) 1609 _rawDataOutput = delegate to accept response data. It should write to the socket or whatever; Cgi does all the needed processing to speak http. 1610 _flush = if _rawDataOutput buffers, this delegate should flush the buffer down the wire 1611 closeConnection = if the request asks to close the connection, *closeConnection == true. 1612 */ 1613 this( 1614 BufferedInputRange inputData, 1615 // string[] headers, immutable(ubyte)[] data, 1616 string address, ushort _port, 1617 int pathInfoStarts = 0, // use this if you know the script name, like if this is in a folder in a bigger web environment 1618 bool _https = false, 1619 void delegate(const(ubyte)[]) _rawDataOutput = null, 1620 void delegate() _flush = null, 1621 // this pointer tells if the connection is supposed to be closed after we handle this 1622 bool* closeConnection = null) 1623 { 1624 // these are all set locally so the loop works 1625 // without triggering errors in dmd 2.064 1626 // we go ahead and set them at the end of it to the this version 1627 int port; 1628 string referrer; 1629 string remoteAddress; 1630 string userAgent; 1631 string authorization; 1632 string origin; 1633 string accept; 1634 string lastEventId; 1635 bool https; 1636 string host; 1637 RequestMethod requestMethod; 1638 string requestUri; 1639 string pathInfo; 1640 string queryString; 1641 string scriptName; 1642 string[string] get; 1643 string[][string] getArray; 1644 bool keepAliveRequested; 1645 bool acceptsGzip; 1646 string cookie; 1647 1648 1649 1650 environmentVariables = cast(const) environment.toAA; 1651 1652 idlol = inputData; 1653 1654 isCalledWithCommandLineArguments = false; 1655 1656 https = _https; 1657 port = _port; 1658 1659 rawDataOutput = _rawDataOutput; 1660 flushDelegate = _flush; 1661 nph = true; 1662 1663 remoteAddress = address; 1664 1665 // streaming parser 1666 import al = std.algorithm; 1667 1668 // FIXME: tis cast is technically wrong, but Phobos deprecated al.indexOf... for some reason. 1669 auto idx = indexOf(cast(string) inputData.front(), "\r\n\r\n"); 1670 while(idx == -1) { 1671 inputData.popFront(0); 1672 idx = indexOf(cast(string) inputData.front(), "\r\n\r\n"); 1673 } 1674 1675 assert(idx != -1); 1676 1677 1678 string contentType = ""; 1679 string[string] requestHeadersHere; 1680 1681 size_t contentLength; 1682 1683 bool isChunked; 1684 1685 { 1686 import core.runtime; 1687 scriptFileName = Runtime.args[0]; 1688 } 1689 1690 1691 int headerNumber = 0; 1692 foreach(line; al.splitter(inputData.front()[0 .. idx], "\r\n")) 1693 if(line.length) { 1694 headerNumber++; 1695 auto header = cast(string) line.idup; 1696 if(headerNumber == 1) { 1697 // request line 1698 auto parts = al.splitter(header, " "); 1699 requestMethod = to!RequestMethod(parts.front); 1700 parts.popFront(); 1701 requestUri = parts.front; 1702 1703 scriptName = requestUri[0 .. pathInfoStarts]; 1704 1705 auto question = requestUri.indexOf("?"); 1706 if(question == -1) { 1707 queryString = ""; 1708 // FIXME: double check, this might be wrong since it could be url encoded 1709 pathInfo = requestUri[pathInfoStarts..$]; 1710 } else { 1711 queryString = requestUri[question+1..$]; 1712 pathInfo = requestUri[pathInfoStarts..question]; 1713 } 1714 1715 auto ugh = decodeVariables(queryString, "&", &allGetNamesInOrder, &allGetValuesInOrder); 1716 getArray = cast(string[][string]) assumeUnique(ugh); 1717 1718 if(header.indexOf("HTTP/1.0") != -1) { 1719 http10 = true; 1720 autoBuffer = true; 1721 if(closeConnection) { 1722 // on http 1.0, close is assumed (unlike http/1.1 where we assume keep alive) 1723 *closeConnection = true; 1724 } 1725 } 1726 } else { 1727 // other header 1728 auto colon = header.indexOf(":"); 1729 if(colon == -1) 1730 throw new Exception("HTTP headers should have a colon!"); 1731 string name = header[0..colon].toLower; 1732 string value = header[colon+2..$]; // skip the colon and the space 1733 1734 requestHeadersHere[name] = value; 1735 1736 if (name == "accept") { 1737 accept = value; 1738 } 1739 else if (name == "origin") { 1740 origin = value; 1741 } 1742 else if (name == "connection") { 1743 if(value == "close" && closeConnection) 1744 *closeConnection = true; 1745 if(value.asLowerCase().canFind("keep-alive")) { 1746 keepAliveRequested = true; 1747 1748 // on http 1.0, the connection is closed by default, 1749 // but not if they request keep-alive. then we don't close 1750 // anymore - undoing the set above 1751 if(http10 && closeConnection) { 1752 *closeConnection = false; 1753 } 1754 } 1755 } 1756 else if (name == "transfer-encoding") { 1757 if(value == "chunked") 1758 isChunked = true; 1759 } 1760 else if (name == "last-event-id") { 1761 lastEventId = value; 1762 } 1763 else if (name == "authorization") { 1764 authorization = value; 1765 } 1766 else if (name == "content-type") { 1767 contentType = value; 1768 } 1769 else if (name == "content-length") { 1770 contentLength = to!size_t(value); 1771 } 1772 else if (name == "x-forwarded-for") { 1773 remoteAddress = value; 1774 } 1775 else if (name == "x-forwarded-host" || name == "host") { 1776 if(name != "host" || host is null) 1777 host = value; 1778 } 1779 // FIXME: https://tools.ietf.org/html/rfc7239 1780 else if (name == "accept-encoding") { 1781 if(value.indexOf("gzip") != -1) 1782 acceptsGzip = true; 1783 } 1784 else if (name == "user-agent") { 1785 userAgent = value; 1786 } 1787 else if (name == "referer") { 1788 referrer = value; 1789 } 1790 else if (name == "cookie") { 1791 cookie ~= value; 1792 } 1793 // else 1794 // ignore it 1795 1796 } 1797 } 1798 1799 inputData.consume(idx + 4); 1800 // done 1801 1802 requestHeaders = assumeUnique(requestHeadersHere); 1803 1804 ByChunkRange dataByChunk; 1805 1806 // reading Content-Length type data 1807 // We need to read up the data we have, and write it out as a chunk. 1808 if(!isChunked) { 1809 dataByChunk = byChunk(inputData, contentLength); 1810 } else { 1811 // chunked requests happen, but not every day. Since we need to know 1812 // the content length (for now, maybe that should change), we'll buffer 1813 // the whole thing here instead of parse streaming. (I think this is what Apache does anyway in cgi modes) 1814 auto data = dechunk(inputData); 1815 1816 // set the range here 1817 dataByChunk = byChunk(data); 1818 contentLength = data.length; 1819 } 1820 1821 assert(dataByChunk !is null); 1822 1823 if(contentLength) { 1824 prepareForIncomingDataChunks(contentType, contentLength); 1825 foreach(dataChunk; dataByChunk) { 1826 handleIncomingDataChunk(dataChunk); 1827 } 1828 postArray = assumeUnique(pps._post); 1829 filesArray = assumeUnique(pps._files); 1830 files = keepLastOf(filesArray); 1831 post = keepLastOf(postArray); 1832 postJson = pps.postJson; 1833 cleanUpPostDataState(); 1834 } 1835 1836 this.port = port; 1837 this.referrer = referrer; 1838 this.remoteAddress = remoteAddress; 1839 this.userAgent = userAgent; 1840 this.authorization = authorization; 1841 this.origin = origin; 1842 this.accept = accept; 1843 this.lastEventId = lastEventId; 1844 this.https = https; 1845 this.host = host; 1846 this.requestMethod = requestMethod; 1847 this.requestUri = requestUri; 1848 this.pathInfo = pathInfo; 1849 this.queryString = queryString; 1850 1851 this.scriptName = scriptName; 1852 this.get = keepLastOf(getArray); 1853 this.getArray = cast(immutable) getArray; 1854 this.keepAliveRequested = keepAliveRequested; 1855 this.acceptsGzip = acceptsGzip; 1856 this.cookie = cookie; 1857 1858 cookiesArray = getCookieArray(); 1859 cookies = keepLastOf(cookiesArray); 1860 1861 } 1862 BufferedInputRange idlol; 1863 1864 private immutable(string[string]) keepLastOf(in string[][string] arr) { 1865 string[string] ca; 1866 foreach(k, v; arr) 1867 ca[k] = v[$-1]; 1868 1869 return assumeUnique(ca); 1870 } 1871 1872 // FIXME duplication 1873 private immutable(UploadedFile[string]) keepLastOf(in UploadedFile[][string] arr) { 1874 UploadedFile[string] ca; 1875 foreach(k, v; arr) 1876 ca[k] = v[$-1]; 1877 1878 return assumeUnique(ca); 1879 } 1880 1881 1882 private immutable(string[][string]) getCookieArray() { 1883 auto forTheLoveOfGod = decodeVariables(cookie, "; "); 1884 return assumeUnique(forTheLoveOfGod); 1885 } 1886 1887 /// Very simple method to require a basic auth username and password. 1888 /// If the http request doesn't include the required credentials, it throws a 1889 /// HTTP 401 error, and an exception. 1890 /// 1891 /// Note: basic auth does not provide great security, especially over unencrypted HTTP; 1892 /// the user's credentials are sent in plain text on every request. 1893 /// 1894 /// If you are using Apache, the HTTP_AUTHORIZATION variable may not be sent to the 1895 /// application. Either use Apache's built in methods for basic authentication, or add 1896 /// something along these lines to your server configuration: 1897 /// 1898 /// RewriteEngine On 1899 /// RewriteCond %{HTTP:Authorization} ^(.*) 1900 /// RewriteRule ^(.*) - [E=HTTP_AUTHORIZATION:%1] 1901 /// 1902 /// To ensure the necessary data is available to cgi.d. 1903 void requireBasicAuth(string user, string pass, string message = null) { 1904 if(authorization != "Basic " ~ Base64.encode(cast(immutable(ubyte)[]) (user ~ ":" ~ pass))) { 1905 setResponseStatus("401 Authorization Required"); 1906 header ("WWW-Authenticate: Basic realm=\""~message~"\""); 1907 close(); 1908 throw new Exception("Not authorized; got " ~ authorization); 1909 } 1910 } 1911 1912 /// Very simple caching controls - setCache(false) means it will never be cached. Good for rapidly updated or sensitive sites. 1913 /// setCache(true) means it will always be cached for as long as possible. Best for static content. 1914 /// Use setResponseExpires and updateResponseExpires for more control 1915 void setCache(bool allowCaching) { 1916 noCache = !allowCaching; 1917 } 1918 1919 /// Set to true and use cgi.write(data, true); to send a gzipped response to browsers 1920 /// who can accept it 1921 bool gzipResponse; 1922 1923 immutable bool acceptsGzip; 1924 immutable bool keepAliveRequested; 1925 1926 /// Set to true if and only if this was initialized with command line arguments 1927 immutable bool isCalledWithCommandLineArguments; 1928 1929 /// This gets a full url for the current request, including port, protocol, host, path, and query 1930 string getCurrentCompleteUri() const { 1931 ushort defaultPort = https ? 443 : 80; 1932 1933 return format("http%s://%s%s%s", 1934 https ? "s" : "", 1935 host, 1936 (!port || port == defaultPort) ? "" : ":" ~ to!string(port), 1937 requestUri); 1938 } 1939 1940 /// You can override this if your site base url isn't the same as the script name 1941 string logicalScriptName() const { 1942 return scriptName; 1943 } 1944 1945 /// Sets the HTTP status of the response. For example, "404 File Not Found" or "500 Internal Server Error". 1946 /// It assumes "200 OK", and automatically changes to "302 Found" if you call setResponseLocation(). 1947 /// Note setResponseStatus() must be called *before* you write() any data to the output. 1948 void setResponseStatus(string status) { 1949 assert(!outputtedResponseData); 1950 responseStatus = status; 1951 } 1952 private string responseStatus = null; 1953 1954 /// Returns true if it is still possible to output headers 1955 bool canOutputHeaders() { 1956 return !isClosed && !outputtedResponseData; 1957 } 1958 1959 /// Sets the location header, which the browser will redirect the user to automatically. 1960 /// Note setResponseLocation() must be called *before* you write() any data to the output. 1961 /// The optional important argument is used if it's a default suggestion rather than something to insist upon. 1962 void setResponseLocation(string uri, bool important = true, string status = null) { 1963 if(!important && isCurrentResponseLocationImportant) 1964 return; // important redirects always override unimportant ones 1965 1966 if(uri is null) { 1967 responseStatus = "200 OK"; 1968 responseLocation = null; 1969 isCurrentResponseLocationImportant = important; 1970 return; // this just cancels the redirect 1971 } 1972 1973 assert(!outputtedResponseData); 1974 if(status is null) 1975 responseStatus = "302 Found"; 1976 else 1977 responseStatus = status; 1978 1979 responseLocation = uri.strip; 1980 isCurrentResponseLocationImportant = important; 1981 } 1982 protected string responseLocation = null; 1983 private bool isCurrentResponseLocationImportant = false; 1984 1985 /// Sets the Expires: http header. See also: updateResponseExpires, setPublicCaching 1986 /// The parameter is in unix_timestamp * 1000. Try setResponseExpires(getUTCtime() + SOME AMOUNT) for normal use. 1987 /// Note: the when parameter is different than setCookie's expire parameter. 1988 void setResponseExpires(long when, bool isPublic = false) { 1989 responseExpires = when; 1990 setCache(true); // need to enable caching so the date has meaning 1991 1992 responseIsPublic = isPublic; 1993 } 1994 private long responseExpires = long.min; 1995 private bool responseIsPublic = false; 1996 1997 /// This is like setResponseExpires, but it can be called multiple times. The setting most in the past is the one kept. 1998 /// If you have multiple functions, they all might call updateResponseExpires about their own return value. The program 1999 /// output as a whole is as cacheable as the least cachable part in the chain. 2000 2001 /// setCache(false) always overrides this - it is, by definition, the strictest anti-cache statement available. If your site outputs sensitive user data, you should probably call setCache(false) when you do, to ensure no other functions will cache the content, as it may be a privacy risk. 2002 /// Conversely, setting here overrides setCache(true), since any expiration date is in the past of infinity. 2003 void updateResponseExpires(long when, bool isPublic) { 2004 if(responseExpires == long.min) 2005 setResponseExpires(when, isPublic); 2006 else if(when < responseExpires) 2007 setResponseExpires(when, responseIsPublic && isPublic); // if any part of it is private, it all is 2008 } 2009 2010 /* 2011 /// Set to true if you want the result to be cached publically - that is, is the content shared? 2012 /// Should generally be false if the user is logged in. It assumes private cache only. 2013 /// setCache(true) also turns on public caching, and setCache(false) sets to private. 2014 void setPublicCaching(bool allowPublicCaches) { 2015 publicCaching = allowPublicCaches; 2016 } 2017 private bool publicCaching = false; 2018 */ 2019 2020 /// Sets an HTTP cookie, automatically encoding the data to the correct string. 2021 /// expiresIn is how many milliseconds in the future the cookie will expire. 2022 /// TIP: to make a cookie accessible from subdomains, set the domain to .yourdomain.com. 2023 /// Note setCookie() must be called *before* you write() any data to the output. 2024 void setCookie(string name, string data, long expiresIn = 0, string path = null, string domain = null, bool httpOnly = false, bool secure = false) { 2025 assert(!outputtedResponseData); 2026 string cookie = std.uri.encodeComponent(name) ~ "="; 2027 cookie ~= std.uri.encodeComponent(data); 2028 if(path !is null) 2029 cookie ~= "; path=" ~ path; 2030 // FIXME: should I just be using max-age here? (also in cache below) 2031 if(expiresIn != 0) 2032 cookie ~= "; expires=" ~ printDate(cast(DateTime) Clock.currTime(UTC()) + dur!"msecs"(expiresIn)); 2033 if(domain !is null) 2034 cookie ~= "; domain=" ~ domain; 2035 if(secure == true) 2036 cookie ~= "; Secure"; 2037 if(httpOnly == true ) 2038 cookie ~= "; HttpOnly"; 2039 2040 if(auto idx = name in cookieIndexes) { 2041 responseCookies[*idx] = cookie; 2042 } else { 2043 cookieIndexes[name] = responseCookies.length; 2044 responseCookies ~= cookie; 2045 } 2046 } 2047 private string[] responseCookies; 2048 private size_t[string] cookieIndexes; 2049 2050 /// Clears a previously set cookie with the given name, path, and domain. 2051 void clearCookie(string name, string path = null, string domain = null) { 2052 assert(!outputtedResponseData); 2053 setCookie(name, "", 1, path, domain); 2054 } 2055 2056 /// Sets the content type of the response, for example "text/html" (the default) for HTML, or "image/png" for a PNG image 2057 void setResponseContentType(string ct) { 2058 assert(!outputtedResponseData); 2059 responseContentType = ct; 2060 } 2061 private string responseContentType = null; 2062 2063 /// Adds a custom header. It should be the name: value, but without any line terminator. 2064 /// For example: header("X-My-Header: Some value"); 2065 /// Note you should use the specialized functions in this object if possible to avoid 2066 /// duplicates in the output. 2067 void header(string h) { 2068 customHeaders ~= h; 2069 } 2070 2071 private string[] customHeaders; 2072 private bool websocketMode; 2073 2074 void flushHeaders(const(void)[] t, bool isAll = false) { 2075 string[] hd; 2076 // Flush the headers 2077 if(responseStatus !is null) { 2078 if(nph) { 2079 if(http10) 2080 hd ~= "HTTP/1.0 " ~ responseStatus; 2081 else 2082 hd ~= "HTTP/1.1 " ~ responseStatus; 2083 } else 2084 hd ~= "Status: " ~ responseStatus; 2085 } else if (nph) { 2086 if(http10) 2087 hd ~= "HTTP/1.0 200 OK"; 2088 else 2089 hd ~= "HTTP/1.1 200 OK"; 2090 } 2091 2092 if(websocketMode) 2093 goto websocket; 2094 2095 if(nph) { // we're responsible for setting the date too according to http 1.1 2096 hd ~= "Date: " ~ printDate(cast(DateTime) Clock.currTime(UTC())); 2097 } 2098 2099 // FIXME: what if the user wants to set his own content-length? 2100 // The custom header function can do it, so maybe that's best. 2101 // Or we could reuse the isAll param. 2102 if(responseLocation !is null) { 2103 hd ~= "Location: " ~ responseLocation; 2104 } 2105 if(!noCache && responseExpires != long.min) { // an explicit expiration date is set 2106 auto expires = SysTime(unixTimeToStdTime(cast(int)(responseExpires / 1000)), UTC()); 2107 hd ~= "Expires: " ~ printDate( 2108 cast(DateTime) expires); 2109 // FIXME: assuming everything is private unless you use nocache - generally right for dynamic pages, but not necessarily 2110 hd ~= "Cache-Control: "~(responseIsPublic ? "public" : "private")~", no-cache=\"set-cookie, set-cookie2\""; 2111 } 2112 if(responseCookies !is null && responseCookies.length > 0) { 2113 foreach(c; responseCookies) 2114 hd ~= "Set-Cookie: " ~ c; 2115 } 2116 if(noCache) { // we specifically do not want caching (this is actually the default) 2117 hd ~= "Cache-Control: private, no-cache=\"set-cookie\""; 2118 hd ~= "Expires: 0"; 2119 hd ~= "Pragma: no-cache"; 2120 } else { 2121 if(responseExpires == long.min) { // caching was enabled, but without a date set - that means assume cache forever 2122 hd ~= "Cache-Control: public"; 2123 hd ~= "Expires: Tue, 31 Dec 2030 14:00:00 GMT"; // FIXME: should not be more than one year in the future 2124 } 2125 } 2126 if(responseContentType !is null) { 2127 hd ~= "Content-Type: " ~ responseContentType; 2128 } else 2129 hd ~= "Content-Type: text/html; charset=utf-8"; 2130 2131 if(gzipResponse && acceptsGzip && isAll) { // FIXME: isAll really shouldn't be necessary 2132 hd ~= "Content-Encoding: gzip"; 2133 } 2134 2135 2136 if(!isAll) { 2137 if(nph && !http10) { 2138 hd ~= "Transfer-Encoding: chunked"; 2139 responseChunked = true; 2140 } 2141 } else { 2142 hd ~= "Content-Length: " ~ to!string(t.length); 2143 if(nph && keepAliveRequested) { 2144 hd ~= "Connection: Keep-Alive"; 2145 } 2146 } 2147 2148 websocket: 2149 if(customHeaders !is null) 2150 hd ~= customHeaders; 2151 2152 // FIXME: what about duplicated headers? 2153 2154 foreach(h; hd) { 2155 if(rawDataOutput !is null) 2156 rawDataOutput(cast(const(ubyte)[]) (h ~ "\r\n")); 2157 else { 2158 version(CRuntime_Musl) { 2159 stdout.rawWrite(h); 2160 stdout.rawWrite("\n"); 2161 } else { 2162 writeln(h); 2163 } 2164 } 2165 } 2166 if(rawDataOutput !is null) 2167 rawDataOutput(cast(const(ubyte)[]) ("\r\n")); 2168 else { 2169 version(CRuntime_Musl) { 2170 stdout.rawWrite("\n"); 2171 } else { 2172 writeln(""); 2173 } 2174 } 2175 2176 outputtedResponseData = true; 2177 } 2178 2179 /// Writes the data to the output, flushing headers if they have not yet been sent. 2180 void write(const(void)[] t, bool isAll = false, bool maybeAutoClose = true) { 2181 assert(!closed, "Output has already been closed"); 2182 2183 if(gzipResponse && acceptsGzip && isAll) { // FIXME: isAll really shouldn't be necessary 2184 // actually gzip the data here 2185 2186 auto c = new Compress(HeaderFormat.gzip); // want gzip 2187 2188 auto data = c.compress(t); 2189 data ~= c.flush(); 2190 2191 // std.file.write("/tmp/last-item", data); 2192 2193 t = data; 2194 } 2195 2196 if(!outputtedResponseData && (!autoBuffer || isAll)) { 2197 flushHeaders(t, isAll); 2198 } 2199 2200 if(requestMethod != RequestMethod.HEAD && t.length > 0) { 2201 if (autoBuffer) { 2202 outputBuffer ~= cast(ubyte[]) t; 2203 } 2204 if(!autoBuffer || isAll) { 2205 if(rawDataOutput !is null) 2206 if(nph && responseChunked) { 2207 //rawDataOutput(makeChunk(cast(const(ubyte)[]) t)); 2208 // we're making the chunk here instead of in a function 2209 // to avoid unneeded gc pressure 2210 rawDataOutput(cast(const(ubyte)[]) toHex(t.length)); 2211 rawDataOutput(cast(const(ubyte)[]) "\r\n"); 2212 rawDataOutput(cast(const(ubyte)[]) t); 2213 rawDataOutput(cast(const(ubyte)[]) "\r\n"); 2214 2215 2216 } else { 2217 rawDataOutput(cast(const(ubyte)[]) t); 2218 } 2219 else 2220 stdout.rawWrite(t); 2221 } 2222 } 2223 2224 if(maybeAutoClose && isAll) 2225 close(); // if you say it is all, that means we're definitely done 2226 // maybeAutoClose can be false though to avoid this (important if you call from inside close()! 2227 } 2228 2229 void flush() { 2230 if(rawDataOutput is null) 2231 stdout.flush(); 2232 else if(flushDelegate !is null) 2233 flushDelegate(); 2234 } 2235 2236 version(autoBuffer) 2237 bool autoBuffer = true; 2238 else 2239 bool autoBuffer = false; 2240 ubyte[] outputBuffer; 2241 2242 /// Flushes the buffers to the network, signifying that you are done. 2243 /// You should always call this explicitly when you are done outputting data. 2244 void close() { 2245 if(closed) 2246 return; // don't double close 2247 2248 if(!outputtedResponseData) 2249 write("", false, false); 2250 2251 // writing auto buffered data 2252 if(requestMethod != RequestMethod.HEAD && autoBuffer) { 2253 if(!nph) 2254 stdout.rawWrite(outputBuffer); 2255 else 2256 write(outputBuffer, true, false); // tell it this is everything 2257 } 2258 2259 // closing the last chunk... 2260 if(nph && rawDataOutput !is null && responseChunked) 2261 rawDataOutput(cast(const(ubyte)[]) "0\r\n\r\n"); 2262 2263 if(flushDelegate) 2264 flushDelegate(); 2265 2266 closed = true; 2267 } 2268 2269 // Closes without doing anything, shouldn't be used often 2270 void rawClose() { 2271 closed = true; 2272 } 2273 2274 /++ 2275 Gets a request variable as a specific type, or the default value of it isn't there 2276 or isn't convertible to the request type. 2277 2278 Checks both GET and POST variables, preferring the POST variable, if available. 2279 2280 A nice trick is using the default value to choose the type: 2281 2282 --- 2283 /* 2284 The return value will match the type of the default. 2285 Here, I gave 10 as a default, so the return value will 2286 be an int. 2287 2288 If the user-supplied value cannot be converted to the 2289 requested type, you will get the default value back. 2290 */ 2291 int a = cgi.request("number", 10); 2292 2293 if(cgi.get["number"] == "11") 2294 assert(a == 11); // conversion succeeds 2295 2296 if("number" !in cgi.get) 2297 assert(a == 10); // no value means you can't convert - give the default 2298 2299 if(cgi.get["number"] == "twelve") 2300 assert(a == 10); // conversion from string to int would fail, so we get the default 2301 --- 2302 2303 You can use an enum as an easy whitelist, too: 2304 2305 --- 2306 enum Operations { 2307 add, remove, query 2308 } 2309 2310 auto op = cgi.request("op", Operations.query); 2311 2312 if(cgi.get["op"] == "add") 2313 assert(op == Operations.add); 2314 if(cgi.get["op"] == "remove") 2315 assert(op == Operations.remove); 2316 if(cgi.get["op"] == "query") 2317 assert(op == Operations.query); 2318 2319 if(cgi.get["op"] == "random string") 2320 assert(op == Operations.query); // the value can't be converted to the enum, so we get the default 2321 --- 2322 +/ 2323 T request(T = string)(in string name, in T def = T.init) const nothrow { 2324 try { 2325 return 2326 (name in post) ? to!T(post[name]) : 2327 (name in get) ? to!T(get[name]) : 2328 def; 2329 } catch(Exception e) { return def; } 2330 } 2331 2332 /// Is the output already closed? 2333 bool isClosed() const { 2334 return closed; 2335 } 2336 2337 /++ 2338 Gets a session object associated with the `cgi` request. You can use different type throughout your application. 2339 +/ 2340 Session!Data getSessionObject(Data)() { 2341 if(testInProcess !is null) { 2342 // test mode 2343 auto obj = testInProcess.getSessionOverride(typeid(typeof(return))); 2344 if(obj !is null) 2345 return cast(typeof(return)) obj; 2346 else { 2347 auto o = new MockSession!Data(); 2348 testInProcess.setSessionOverride(typeid(typeof(return)), o); 2349 return o; 2350 } 2351 } else { 2352 // normal operation 2353 return new BasicDataServerSession!Data(this); 2354 } 2355 } 2356 2357 // if it is in test mode; triggers mock sessions. Used by CgiTester 2358 version(with_breaking_cgi_features) 2359 private CgiTester testInProcess; 2360 2361 /* Hooks for redirecting input and output */ 2362 private void delegate(const(ubyte)[]) rawDataOutput = null; 2363 private void delegate() flushDelegate = null; 2364 2365 /* This info is used when handling a more raw HTTP protocol */ 2366 private bool nph; 2367 private bool http10; 2368 private bool closed; 2369 private bool responseChunked = false; 2370 2371 version(preserveData) // note: this can eat lots of memory; don't use unless you're sure you need it. 2372 immutable(ubyte)[] originalPostData; 2373 2374 public immutable string postJson; 2375 2376 /* Internal state flags */ 2377 private bool outputtedResponseData; 2378 private bool noCache = true; 2379 2380 const(string[string]) environmentVariables; 2381 2382 /** What follows is data gotten from the HTTP request. It is all fully immutable, 2383 partially because it logically is (your code doesn't change what the user requested...) 2384 and partially because I hate how bad programs in PHP change those superglobals to do 2385 all kinds of hard to follow ugliness. I don't want that to ever happen in D. 2386 2387 For some of these, you'll want to refer to the http or cgi specs for more details. 2388 */ 2389 immutable(string[string]) requestHeaders; /// All the raw headers in the request as name/value pairs. The name is stored as all lower case, but otherwise the same as it is in HTTP; words separated by dashes. For example, "cookie" or "accept-encoding". Many HTTP headers have specialized variables below for more convenience and static name checking; you should generally try to use them. 2390 2391 immutable(char[]) host; /// The hostname in the request. If one program serves multiple domains, you can use this to differentiate between them. 2392 immutable(char[]) origin; /// The origin header in the request, if present. Some HTML5 cross-domain apis set this and you should check it on those cross domain requests and websockets. 2393 immutable(char[]) userAgent; /// The browser's user-agent string. Can be used to identify the browser. 2394 immutable(char[]) pathInfo; /// This is any stuff sent after your program's name on the url, but before the query string. For example, suppose your program is named "app". If the user goes to site.com/app, pathInfo is empty. But, he can also go to site.com/app/some/sub/path; treating your program like a virtual folder. In this case, pathInfo == "/some/sub/path". 2395 immutable(char[]) scriptName; /// The full base path of your program, as seen by the user. If your program is located at site.com/programs/apps, scriptName == "/programs/apps". 2396 immutable(char[]) scriptFileName; /// The physical filename of your script 2397 immutable(char[]) authorization; /// The full authorization string from the header, undigested. Useful for implementing auth schemes such as OAuth 1.0. Note that some web servers do not forward this to the app without taking extra steps. See requireBasicAuth's comment for more info. 2398 immutable(char[]) accept; /// The HTTP accept header is the user agent telling what content types it is willing to accept. This is often */*; they accept everything, so it's not terribly useful. (The similar sounding Accept-Encoding header is handled automatically for chunking and gzipping. Simply set gzipResponse = true and cgi.d handles the details, zipping if the user's browser is willing to accept it.) 2399 immutable(char[]) lastEventId; /// The HTML 5 draft includes an EventSource() object that connects to the server, and remains open to take a stream of events. My arsd.rtud module can help with the server side part of that. The Last-Event-Id http header is defined in the draft to help handle loss of connection. When the browser reconnects to you, it sets this header to the last event id it saw, so you can catch it up. This member has the contents of that header. 2400 2401 immutable(RequestMethod) requestMethod; /// The HTTP request verb: GET, POST, etc. It is represented as an enum in cgi.d (which, like many enums, you can convert back to string with std.conv.to()). A HTTP GET is supposed to, according to the spec, not have side effects; a user can GET something over and over again and always have the same result. On all requests, the get[] and getArray[] members may be filled in. The post[] and postArray[] members are only filled in on POST methods. 2402 immutable(char[]) queryString; /// The unparsed content of the request query string - the stuff after the ? in your URL. See get[] and getArray[] for a parse view of it. Sometimes, the unparsed string is useful though if you want a custom format of data up there (probably not a good idea, unless it is really simple, like "?username" perhaps.) 2403 immutable(char[]) cookie; /// The unparsed content of the Cookie: header in the request. See also the cookies[string] member for a parsed view of the data. 2404 /** The Referer header from the request. (It is misspelled in the HTTP spec, and thus the actual request and cgi specs too, but I spelled the word correctly here because that's sane. The spec's misspelling is an implementation detail.) It contains the site url that referred the user to your program; the site that linked to you, or if you're serving images, the site that has you as an image. Also, if you're in an iframe, the referrer is the site that is framing you. 2405 2406 Important note: if the user copy/pastes your url, this is blank, and, just like with all other user data, their browsers can also lie to you. Don't rely on it for real security. 2407 */ 2408 immutable(char[]) referrer; 2409 immutable(char[]) requestUri; /// The full url if the current request, excluding the protocol and host. requestUri == scriptName ~ pathInfo ~ (queryString.length ? "?" ~ queryString : ""); 2410 2411 immutable(char[]) remoteAddress; /// The IP address of the user, as we see it. (Might not match the IP of the user's computer due to things like proxies and NAT.) 2412 2413 immutable bool https; /// Was the request encrypted via https? 2414 immutable int port; /// On what TCP port number did the server receive the request? 2415 2416 /** Here come the parsed request variables - the things that come close to PHP's _GET, _POST, etc. superglobals in content. */ 2417 2418 immutable(string[string]) get; /// The data from your query string in the url, only showing the last string of each name. If you want to handle multiple values with the same name, use getArray. This only works right if the query string is x-www-form-urlencoded; the default you see on the web with name=value pairs separated by the & character. 2419 immutable(string[string]) post; /// The data from the request's body, on POST requests. It parses application/x-www-form-urlencoded data (used by most web requests, including typical forms), and multipart/form-data requests (used by file uploads on web forms) into the same container, so you can always access them the same way. It makes no attempt to parse other content types. If you want to accept an XML Post body (for a web api perhaps), you'll need to handle the raw data yourself. 2420 immutable(string[string]) cookies; /// Separates out the cookie header into individual name/value pairs (which is how you set them!) 2421 2422 /** 2423 Represents user uploaded files. 2424 2425 When making a file upload form, be sure to follow the standard: set method="POST" and enctype="multipart/form-data" in your html <form> tag attributes. The key into this array is the name attribute on your input tag, just like with other post variables. See the comments on the UploadedFile struct for more information about the data inside, including important notes on max size and content location. 2426 */ 2427 immutable(UploadedFile[][string]) filesArray; 2428 immutable(UploadedFile[string]) files; 2429 2430 /// Use these if you expect multiple items submitted with the same name. btw, assert(get[name] is getArray[name][$-1); should pass. Same for post and cookies. 2431 /// the order of the arrays is the order the data arrives 2432 immutable(string[][string]) getArray; /// like get, but an array of values per name 2433 immutable(string[][string]) postArray; /// ditto for post 2434 immutable(string[][string]) cookiesArray; /// ditto for cookies 2435 2436 // convenience function for appending to a uri without extra ? 2437 // matches the name and effect of javascript's location.search property 2438 string search() const { 2439 if(queryString.length) 2440 return "?" ~ queryString; 2441 return ""; 2442 } 2443 2444 // FIXME: what about multiple files with the same name? 2445 private: 2446 //RequestMethod _requestMethod; 2447 } 2448 2449 /// use this for testing or other isolated things when you want it to be no-ops 2450 Cgi dummyCgi(Cgi.RequestMethod method = Cgi.RequestMethod.GET, string url = null, in ubyte[] data = null, void delegate(const(ubyte)[]) outputSink = null) { 2451 // we want to ignore, not use stdout 2452 if(outputSink is null) 2453 outputSink = delegate void(const(ubyte)[]) { }; 2454 2455 string[string] env; 2456 env["REQUEST_METHOD"] = to!string(method); 2457 env["CONTENT_LENGTH"] = to!string(data.length); 2458 2459 auto cgi = new Cgi( 2460 0, 2461 env, 2462 { return data; }, 2463 outputSink, 2464 null); 2465 2466 return cgi; 2467 } 2468 2469 /++ 2470 A helper test class for request handler unittests. 2471 +/ 2472 version(with_breaking_cgi_features) 2473 class CgiTester { 2474 private { 2475 SessionObject[TypeInfo] mockSessions; 2476 SessionObject getSessionOverride(TypeInfo ti) { 2477 if(auto o = ti in mockSessions) 2478 return *o; 2479 else 2480 return null; 2481 } 2482 void setSessionOverride(TypeInfo ti, SessionObject so) { 2483 mockSessions[ti] = so; 2484 } 2485 } 2486 2487 /++ 2488 Gets (and creates if necessary) a mock session object for this test. Note 2489 it will be the same one used for any test operations through this CgiTester instance. 2490 +/ 2491 Session!Data getSessionObject(Data)() { 2492 auto obj = getSessionOverride(typeid(typeof(return))); 2493 if(obj !is null) 2494 return cast(typeof(return)) obj; 2495 else { 2496 auto o = new MockSession!Data(); 2497 setSessionOverride(typeid(typeof(return)), o); 2498 return o; 2499 } 2500 } 2501 2502 /++ 2503 Pass a reference to your request handler when creating the tester. 2504 +/ 2505 this(void function(Cgi) requestHandler) { 2506 this.requestHandler = requestHandler; 2507 } 2508 2509 /++ 2510 You can check response information with these methods after you call the request handler. 2511 +/ 2512 struct Response { 2513 int code; 2514 string[string] headers; 2515 string responseText; 2516 ubyte[] responseBody; 2517 } 2518 2519 /++ 2520 Executes a test request on your request handler, and returns the response. 2521 2522 Params: 2523 url = The URL to test. Should be an absolute path, but excluding domain. e.g. `"/test"`. 2524 args = additional arguments. Same format as cgi's command line handler. 2525 +/ 2526 Response GET(string url, string[] args = null) { 2527 return executeTest("GET", url, args); 2528 } 2529 /// ditto 2530 Response POST(string url, string[] args = null) { 2531 return executeTest("POST", url, args); 2532 } 2533 2534 /// ditto 2535 Response executeTest(string method, string url, string[] args) { 2536 ubyte[] outputtedRawData; 2537 void outputSink(const(ubyte)[] data) { 2538 outputtedRawData ~= data; 2539 } 2540 auto cgi = new Cgi(["test", method, url] ~ args, &outputSink); 2541 cgi.testInProcess = this; 2542 scope(exit) cgi.dispose(); 2543 2544 requestHandler(cgi); 2545 2546 cgi.close(); 2547 2548 Response response; 2549 2550 if(outputtedRawData.length) { 2551 enum LINE = "\r\n"; 2552 2553 auto idx = outputtedRawData.locationOf(LINE ~ LINE); 2554 assert(idx != -1, to!string(outputtedRawData)); 2555 auto headers = cast(string) outputtedRawData[0 .. idx]; 2556 response.code = 200; 2557 while(headers.length) { 2558 auto i = headers.locationOf(LINE); 2559 if(i == -1) i = cast(int) headers.length; 2560 2561 auto header = headers[0 .. i]; 2562 2563 auto c = header.locationOf(":"); 2564 if(c != -1) { 2565 auto name = header[0 .. c]; 2566 auto value = header[c + 2 ..$]; 2567 2568 if(name == "Status") 2569 response.code = value[0 .. value.locationOf(" ")].to!int; 2570 2571 response.headers[name] = value; 2572 } else { 2573 assert(0); 2574 } 2575 2576 if(i != headers.length) 2577 i += 2; 2578 headers = headers[i .. $]; 2579 } 2580 response.responseBody = outputtedRawData[idx + 4 .. $]; 2581 response.responseText = cast(string) response.responseBody; 2582 } 2583 2584 return response; 2585 } 2586 2587 private void function(Cgi) requestHandler; 2588 } 2589 2590 2591 // should this be a separate module? Probably, but that's a hassle. 2592 2593 /// Makes a data:// uri that can be used as links in most newer browsers (IE8+). 2594 string makeDataUrl(string mimeType, in void[] data) { 2595 auto data64 = Base64.encode(cast(const(ubyte[])) data); 2596 return "data:" ~ mimeType ~ ";base64," ~ assumeUnique(data64); 2597 } 2598 2599 // FIXME: I don't think this class correctly decodes/encodes the individual parts 2600 /// Represents a url that can be broken down or built up through properties 2601 struct Uri { 2602 alias toString this; // blargh idk a url really is a string, but should it be implicit? 2603 2604 // scheme//userinfo@host:port/path?query#fragment 2605 2606 string scheme; /// e.g. "http" in "http://example.com/" 2607 string userinfo; /// the username (and possibly a password) in the uri 2608 string host; /// the domain name 2609 int port; /// port number, if given. Will be zero if a port was not explicitly given 2610 string path; /// e.g. "/folder/file.html" in "http://example.com/folder/file.html" 2611 string query; /// the stuff after the ? in a uri 2612 string fragment; /// the stuff after the # in a uri. 2613 2614 // idk if i want to keep these, since the functions they wrap are used many, many, many times in existing code, so this is either an unnecessary alias or a gratuitous break of compatibility 2615 // the decode ones need to keep different names anyway because we can't overload on return values... 2616 static string encode(string s) { return std.uri.encodeComponent(s); } 2617 static string encode(string[string] s) { return encodeVariables(s); } 2618 static string encode(string[][string] s) { return encodeVariables(s); } 2619 2620 /// Breaks down a uri string to its components 2621 this(string uri) { 2622 reparse(uri); 2623 } 2624 2625 private void reparse(string uri) { 2626 // from RFC 3986 2627 // the ctRegex triples the compile time and makes ugly errors for no real benefit 2628 // it was a nice experiment but just not worth it. 2629 // enum ctr = ctRegex!r"^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?"; 2630 /* 2631 Captures: 2632 0 = whole url 2633 1 = scheme, with : 2634 2 = scheme, no : 2635 3 = authority, with // 2636 4 = authority, no // 2637 5 = path 2638 6 = query string, with ? 2639 7 = query string, no ? 2640 8 = anchor, with # 2641 9 = anchor, no # 2642 */ 2643 // Yikes, even regular, non-CT regex is also unacceptably slow to compile. 1.9s on my computer! 2644 // instead, I will DIY and cut that down to 0.6s on the same computer. 2645 /* 2646 2647 Note that authority is 2648 user:password@domain:port 2649 where the user:password@ part is optional, and the :port is optional. 2650 2651 Regex translation: 2652 2653 Scheme cannot have :, /, ?, or # in it, and must have one or more chars and end in a :. It is optional, but must be first. 2654 Authority must start with //, but cannot have any other /, ?, or # in it. It is optional. 2655 Path cannot have any ? or # in it. It is optional. 2656 Query must start with ? and must not have # in it. It is optional. 2657 Anchor must start with # and can have anything else in it to end of string. It is optional. 2658 */ 2659 2660 this = Uri.init; // reset all state 2661 2662 // empty uri = nothing special 2663 if(uri.length == 0) { 2664 return; 2665 } 2666 2667 size_t idx; 2668 2669 scheme_loop: foreach(char c; uri[idx .. $]) { 2670 switch(c) { 2671 case ':': 2672 case '/': 2673 case '?': 2674 case '#': 2675 break scheme_loop; 2676 default: 2677 } 2678 idx++; 2679 } 2680 2681 if(idx == 0 && uri[idx] == ':') { 2682 // this is actually a path! we skip way ahead 2683 goto path_loop; 2684 } 2685 2686 if(idx == uri.length) { 2687 // the whole thing is a path, apparently 2688 path = uri; 2689 return; 2690 } 2691 2692 if(idx > 0 && uri[idx] == ':') { 2693 scheme = uri[0 .. idx]; 2694 idx++; 2695 } else { 2696 // we need to rewind; it found a / but no :, so the whole thing is prolly a path... 2697 idx = 0; 2698 } 2699 2700 if(idx + 2 < uri.length && uri[idx .. idx + 2] == "//") { 2701 // we have an authority.... 2702 idx += 2; 2703 2704 auto authority_start = idx; 2705 authority_loop: foreach(char c; uri[idx .. $]) { 2706 switch(c) { 2707 case '/': 2708 case '?': 2709 case '#': 2710 break authority_loop; 2711 default: 2712 } 2713 idx++; 2714 } 2715 2716 auto authority = uri[authority_start .. idx]; 2717 2718 auto idx2 = authority.indexOf("@"); 2719 if(idx2 != -1) { 2720 userinfo = authority[0 .. idx2]; 2721 authority = authority[idx2 + 1 .. $]; 2722 } 2723 2724 idx2 = authority.indexOf(":"); 2725 if(idx2 == -1) { 2726 port = 0; // 0 means not specified; we should use the default for the scheme 2727 host = authority; 2728 } else { 2729 host = authority[0 .. idx2]; 2730 port = to!int(authority[idx2 + 1 .. $]); 2731 } 2732 } 2733 2734 path_loop: 2735 auto path_start = idx; 2736 2737 foreach(char c; uri[idx .. $]) { 2738 if(c == '?' || c == '#') 2739 break; 2740 idx++; 2741 } 2742 2743 path = uri[path_start .. idx]; 2744 2745 if(idx == uri.length) 2746 return; // nothing more to examine... 2747 2748 if(uri[idx] == '?') { 2749 idx++; 2750 auto query_start = idx; 2751 foreach(char c; uri[idx .. $]) { 2752 if(c == '#') 2753 break; 2754 idx++; 2755 } 2756 query = uri[query_start .. idx]; 2757 } 2758 2759 if(idx < uri.length && uri[idx] == '#') { 2760 idx++; 2761 fragment = uri[idx .. $]; 2762 } 2763 2764 // uriInvalidated = false; 2765 } 2766 2767 private string rebuildUri() const { 2768 string ret; 2769 if(scheme.length) 2770 ret ~= scheme ~ ":"; 2771 if(userinfo.length || host.length) 2772 ret ~= "//"; 2773 if(userinfo.length) 2774 ret ~= userinfo ~ "@"; 2775 if(host.length) 2776 ret ~= host; 2777 if(port) 2778 ret ~= ":" ~ to!string(port); 2779 2780 ret ~= path; 2781 2782 if(query.length) 2783 ret ~= "?" ~ query; 2784 2785 if(fragment.length) 2786 ret ~= "#" ~ fragment; 2787 2788 // uri = ret; 2789 // uriInvalidated = false; 2790 return ret; 2791 } 2792 2793 /// Converts the broken down parts back into a complete string 2794 string toString() const { 2795 // if(uriInvalidated) 2796 return rebuildUri(); 2797 } 2798 2799 /// Returns a new absolute Uri given a base. It treats this one as 2800 /// relative where possible, but absolute if not. (If protocol, domain, or 2801 /// other info is not set, the new one inherits it from the base.) 2802 /// 2803 /// Browsers use a function like this to figure out links in html. 2804 Uri basedOn(in Uri baseUrl) const { 2805 Uri n = this; // copies 2806 // n.uriInvalidated = true; // make sure we regenerate... 2807 2808 // userinfo is not inherited... is this wrong? 2809 2810 // if anything is given in the existing url, we don't use the base anymore. 2811 if(n.scheme.empty) { 2812 n.scheme = baseUrl.scheme; 2813 if(n.host.empty) { 2814 n.host = baseUrl.host; 2815 if(n.port == 0) { 2816 n.port = baseUrl.port; 2817 if(n.path.length > 0 && n.path[0] != '/') { 2818 auto b = baseUrl.path[0 .. baseUrl.path.lastIndexOf("/") + 1]; 2819 if(b.length == 0) 2820 b = "/"; 2821 n.path = b ~ n.path; 2822 } else if(n.path.length == 0) { 2823 n.path = baseUrl.path; 2824 } 2825 } 2826 } 2827 } 2828 2829 n.removeDots(); 2830 2831 return n; 2832 } 2833 2834 void removeDots() { 2835 auto parts = this.path.split("/"); 2836 string[] toKeep; 2837 foreach(part; parts) { 2838 if(part == ".") { 2839 continue; 2840 } else if(part == "..") { 2841 toKeep = toKeep[0 .. $-1]; 2842 continue; 2843 } else { 2844 toKeep ~= part; 2845 } 2846 } 2847 2848 this.path = toKeep.join("/"); 2849 } 2850 2851 unittest { 2852 auto uri = Uri("test.html"); 2853 assert(uri.path == "test.html"); 2854 uri = Uri("path/1/lol"); 2855 assert(uri.path == "path/1/lol"); 2856 uri = Uri("http://me@example.com"); 2857 assert(uri.scheme == "http"); 2858 assert(uri.userinfo == "me"); 2859 assert(uri.host == "example.com"); 2860 uri = Uri("http://example.com/#a"); 2861 assert(uri.scheme == "http"); 2862 assert(uri.host == "example.com"); 2863 assert(uri.fragment == "a"); 2864 uri = Uri("#foo"); 2865 assert(uri.fragment == "foo"); 2866 uri = Uri("?lol"); 2867 assert(uri.query == "lol"); 2868 uri = Uri("#foo?lol"); 2869 assert(uri.fragment == "foo?lol"); 2870 uri = Uri("?lol#foo"); 2871 assert(uri.fragment == "foo"); 2872 assert(uri.query == "lol"); 2873 } 2874 2875 // This can sometimes be a big pain in the butt for me, so lots of copy/paste here to cover 2876 // the possibilities. 2877 unittest { 2878 auto url = Uri("cool.html"); // checking relative links 2879 2880 assert(url.basedOn(Uri("http://test.com/what/test.html")) == "http://test.com/what/cool.html"); 2881 assert(url.basedOn(Uri("https://test.com/what/test.html")) == "https://test.com/what/cool.html"); 2882 assert(url.basedOn(Uri("http://test.com/what/")) == "http://test.com/what/cool.html"); 2883 assert(url.basedOn(Uri("http://test.com/")) == "http://test.com/cool.html"); 2884 assert(url.basedOn(Uri("http://test.com")) == "http://test.com/cool.html"); 2885 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b")) == "http://test.com/what/cool.html"); 2886 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d")) == "http://test.com/what/cool.html"); 2887 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d#what")) == "http://test.com/what/cool.html"); 2888 assert(url.basedOn(Uri("http://test.com")) == "http://test.com/cool.html"); 2889 2890 url = Uri("/something/cool.html"); // same server, different path 2891 assert(url.basedOn(Uri("http://test.com/what/test.html")) == "http://test.com/something/cool.html"); 2892 assert(url.basedOn(Uri("https://test.com/what/test.html")) == "https://test.com/something/cool.html"); 2893 assert(url.basedOn(Uri("http://test.com/what/")) == "http://test.com/something/cool.html"); 2894 assert(url.basedOn(Uri("http://test.com/")) == "http://test.com/something/cool.html"); 2895 assert(url.basedOn(Uri("http://test.com")) == "http://test.com/something/cool.html"); 2896 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b")) == "http://test.com/something/cool.html"); 2897 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d")) == "http://test.com/something/cool.html"); 2898 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d#what")) == "http://test.com/something/cool.html"); 2899 assert(url.basedOn(Uri("http://test.com")) == "http://test.com/something/cool.html"); 2900 2901 url = Uri("?query=answer"); // same path. server, protocol, and port, just different query string and fragment 2902 assert(url.basedOn(Uri("http://test.com/what/test.html")) == "http://test.com/what/test.html?query=answer"); 2903 assert(url.basedOn(Uri("https://test.com/what/test.html")) == "https://test.com/what/test.html?query=answer"); 2904 assert(url.basedOn(Uri("http://test.com/what/")) == "http://test.com/what/?query=answer"); 2905 assert(url.basedOn(Uri("http://test.com/")) == "http://test.com/?query=answer"); 2906 assert(url.basedOn(Uri("http://test.com")) == "http://test.com?query=answer"); 2907 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b")) == "http://test.com/what/test.html?query=answer"); 2908 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d")) == "http://test.com/what/test.html?query=answer"); 2909 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d#what")) == "http://test.com/what/test.html?query=answer"); 2910 assert(url.basedOn(Uri("http://test.com")) == "http://test.com?query=answer"); 2911 2912 url = Uri("/test/bar"); 2913 assert(Uri("./").basedOn(url) == "/test/", Uri("./").basedOn(url)); 2914 assert(Uri("../").basedOn(url) == "/"); 2915 2916 //auto uriBefore = url; 2917 url = Uri("#anchor"); // everything should remain the same except the anchor 2918 //uriBefore.anchor = "anchor"); 2919 //assert(url == uriBefore); 2920 2921 url = Uri("//example.com"); // same protocol, but different server. the path here should be blank. 2922 2923 url = Uri("//example.com/example.html"); // same protocol, but different server and path 2924 2925 url = Uri("http://example.com/test.html"); // completely absolute link should never be modified 2926 2927 url = Uri("http://example.com"); // completely absolute link should never be modified, even if it has no path 2928 2929 // FIXME: add something for port too 2930 } 2931 2932 // these are like javascript's location.search and location.hash 2933 string search() const { 2934 return query.length ? ("?" ~ query) : ""; 2935 } 2936 string hash() const { 2937 return fragment.length ? ("#" ~ fragment) : ""; 2938 } 2939 } 2940 2941 2942 /* 2943 for session, see web.d 2944 */ 2945 2946 /// breaks down a url encoded string 2947 string[][string] decodeVariables(string data, string separator = "&", string[]* namesInOrder = null, string[]* valuesInOrder = null) { 2948 auto vars = data.split(separator); 2949 string[][string] _get; 2950 foreach(var; vars) { 2951 auto equal = var.indexOf("="); 2952 string name; 2953 string value; 2954 if(equal == -1) { 2955 name = decodeComponent(var); 2956 value = ""; 2957 } else { 2958 //_get[decodeComponent(var[0..equal])] ~= decodeComponent(var[equal + 1 .. $].replace("+", " ")); 2959 // stupid + -> space conversion. 2960 name = decodeComponent(var[0..equal].replace("+", " ")); 2961 value = decodeComponent(var[equal + 1 .. $].replace("+", " ")); 2962 } 2963 2964 _get[name] ~= value; 2965 if(namesInOrder) 2966 (*namesInOrder) ~= name; 2967 if(valuesInOrder) 2968 (*valuesInOrder) ~= value; 2969 } 2970 return _get; 2971 } 2972 2973 /// breaks down a url encoded string, but only returns the last value of any array 2974 string[string] decodeVariablesSingle(string data) { 2975 string[string] va; 2976 auto varArray = decodeVariables(data); 2977 foreach(k, v; varArray) 2978 va[k] = v[$-1]; 2979 2980 return va; 2981 } 2982 2983 /// url encodes the whole string 2984 string encodeVariables(in string[string] data) { 2985 string ret; 2986 2987 bool outputted = false; 2988 foreach(k, v; data) { 2989 if(outputted) 2990 ret ~= "&"; 2991 else 2992 outputted = true; 2993 2994 ret ~= std.uri.encodeComponent(k) ~ "=" ~ std.uri.encodeComponent(v); 2995 } 2996 2997 return ret; 2998 } 2999 3000 /// url encodes a whole string 3001 string encodeVariables(in string[][string] data) { 3002 string ret; 3003 3004 bool outputted = false; 3005 foreach(k, arr; data) { 3006 foreach(v; arr) { 3007 if(outputted) 3008 ret ~= "&"; 3009 else 3010 outputted = true; 3011 ret ~= std.uri.encodeComponent(k) ~ "=" ~ std.uri.encodeComponent(v); 3012 } 3013 } 3014 3015 return ret; 3016 } 3017 3018 /// Encodes all but the explicitly unreserved characters per rfc 3986 3019 /// Alphanumeric and -_.~ are the only ones left unencoded 3020 /// name is borrowed from php 3021 string rawurlencode(in char[] data) { 3022 string ret; 3023 ret.reserve(data.length * 2); 3024 foreach(char c; data) { 3025 if( 3026 (c >= 'a' && c <= 'z') || 3027 (c >= 'A' && c <= 'Z') || 3028 (c >= '0' && c <= '9') || 3029 c == '-' || c == '_' || c == '.' || c == '~') 3030 { 3031 ret ~= c; 3032 } else { 3033 ret ~= '%'; 3034 // since we iterate on char, this should give us the octets of the full utf8 string 3035 ret ~= toHexUpper(c); 3036 } 3037 } 3038 3039 return ret; 3040 } 3041 3042 3043 // http helper functions 3044 3045 // for chunked responses (which embedded http does whenever possible) 3046 version(none) // this is moved up above to avoid making a copy of the data 3047 const(ubyte)[] makeChunk(const(ubyte)[] data) { 3048 const(ubyte)[] ret; 3049 3050 ret = cast(const(ubyte)[]) toHex(data.length); 3051 ret ~= cast(const(ubyte)[]) "\r\n"; 3052 ret ~= data; 3053 ret ~= cast(const(ubyte)[]) "\r\n"; 3054 3055 return ret; 3056 } 3057 3058 string toHex(long num) { 3059 string ret; 3060 while(num) { 3061 int v = num % 16; 3062 num /= 16; 3063 char d = cast(char) ((v < 10) ? v + '0' : (v-10) + 'a'); 3064 ret ~= d; 3065 } 3066 3067 return to!string(array(ret.retro)); 3068 } 3069 3070 string toHexUpper(long num) { 3071 string ret; 3072 while(num) { 3073 int v = num % 16; 3074 num /= 16; 3075 char d = cast(char) ((v < 10) ? v + '0' : (v-10) + 'A'); 3076 ret ~= d; 3077 } 3078 3079 if(ret.length == 1) 3080 ret ~= "0"; // url encoding requires two digits and that's what this function is used for... 3081 3082 return to!string(array(ret.retro)); 3083 } 3084 3085 3086 // the generic mixins 3087 3088 /// Use this instead of writing your own main 3089 mixin template GenericMain(alias fun, long maxContentLength = defaultMaxContentLength) { 3090 mixin CustomCgiMain!(Cgi, fun, maxContentLength); 3091 } 3092 3093 private string simpleHtmlEncode(string s) { 3094 return s.replace("&", "&").replace("<", "<").replace(">", ">").replace("\n", "<br />\n"); 3095 } 3096 3097 string messageFromException(Throwable t) { 3098 string message; 3099 if(t !is null) { 3100 debug message = t.toString(); 3101 else message = "An unexpected error has occurred."; 3102 } else { 3103 message = "Unknown error"; 3104 } 3105 return message; 3106 } 3107 3108 string plainHttpError(bool isCgi, string type, Throwable t) { 3109 auto message = messageFromException(t); 3110 message = simpleHtmlEncode(message); 3111 3112 return format("%s %s\r\nContent-Length: %s\r\n\r\n%s", 3113 isCgi ? "Status:" : "HTTP/1.0", 3114 type, message.length, message); 3115 } 3116 3117 // returns true if we were able to recover reasonably 3118 bool handleException(Cgi cgi, Throwable t) { 3119 if(cgi.isClosed) { 3120 // if the channel has been explicitly closed, we can't handle it here 3121 return true; 3122 } 3123 3124 if(cgi.outputtedResponseData) { 3125 // the headers are sent, but the channel is open... since it closes if all was sent, we can append an error message here. 3126 return false; // but I don't want to, since I don't know what condition the output is in; I don't want to inject something (nor check the content-type for that matter. So we say it was not a clean handling. 3127 } else { 3128 // no headers are sent, we can send a full blown error and recover 3129 cgi.setCache(false); 3130 cgi.setResponseContentType("text/html"); 3131 cgi.setResponseLocation(null); // cancel the redirect 3132 cgi.setResponseStatus("500 Internal Server Error"); 3133 cgi.write(simpleHtmlEncode(messageFromException(t))); 3134 cgi.close(); 3135 return true; 3136 } 3137 } 3138 3139 bool isCgiRequestMethod(string s) { 3140 s = s.toUpper(); 3141 if(s == "COMMANDLINE") 3142 return true; 3143 foreach(member; __traits(allMembers, Cgi.RequestMethod)) 3144 if(s == member) 3145 return true; 3146 return false; 3147 } 3148 3149 /// If you want to use a subclass of Cgi with generic main, use this mixin. 3150 mixin template CustomCgiMain(CustomCgi, alias fun, long maxContentLength = defaultMaxContentLength) if(is(CustomCgi : Cgi)) { 3151 // kinda hacky - the T... is passed to Cgi's constructor in standard cgi mode, and ignored elsewhere 3152 void main(string[] args) { 3153 cgiMainImpl!(fun, CustomCgi, maxContentLength)(args); 3154 } 3155 } 3156 3157 version(embedded_httpd_processes) 3158 int processPoolSize = 8; 3159 3160 void cgiMainImpl(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)(string[] args) if(is(CustomCgi : Cgi)) { 3161 if(args.length > 1) { 3162 // run the special separate processes if needed 3163 switch(args[1]) { 3164 case "--websocket-server": 3165 version(with_addon_servers) 3166 runWebsocketServer(); 3167 else 3168 printf("Add-on servers not compiled in."); 3169 return; 3170 case "--session-server": 3171 version(with_addon_servers) 3172 runSessionServer(); 3173 else 3174 printf("Add-on servers not compiled in."); 3175 return; 3176 case "--event-server": 3177 version(with_addon_servers) 3178 runEventServer(); 3179 else 3180 printf("Add-on servers not compiled in."); 3181 return; 3182 case "--timer-server": 3183 version(with_addon_servers) 3184 runTimerServer(); 3185 else 3186 printf("Add-on servers not compiled in."); 3187 return; 3188 case "--timed-jobs": 3189 import core.demangle; 3190 version(with_addon_servers_connections) 3191 foreach(k, v; scheduledJobHandlers) 3192 writeln(k, "\t", demangle(k)); 3193 return; 3194 case "--timed-job": 3195 scheduledJobHandlers[args[2]](args[3 .. $]); 3196 return; 3197 default: 3198 // intentionally blank - do nothing and carry on to run normally 3199 } 3200 } 3201 3202 // we support command line thing for easy testing everywhere 3203 // it needs to be called ./app method uri [other args...] 3204 if(args.length >= 3 && isCgiRequestMethod(args[1])) { 3205 Cgi cgi = new CustomCgi(args); 3206 scope(exit) cgi.dispose(); 3207 fun(cgi); 3208 cgi.close(); 3209 return; 3210 } 3211 3212 3213 ushort listeningPort(ushort def) { 3214 bool found = false; 3215 foreach(arg; args) { 3216 if(found) 3217 return to!ushort(arg); 3218 if(arg == "--port" || arg == "-p" || arg == "/port" || arg == "--listening-port") 3219 found = true; 3220 } 3221 return def; 3222 } 3223 3224 string listeningHost() { 3225 bool found = false; 3226 foreach(arg; args) { 3227 if(found) 3228 return arg; 3229 if(arg == "--listening-host" || arg == "-h" || arg == "/listening-host") 3230 found = true; 3231 } 3232 return ""; 3233 } 3234 version(netman_httpd) { 3235 import arsd.httpd; 3236 // what about forwarding the other constructor args? 3237 // this probably needs a whole redoing... 3238 serveHttp!CustomCgi(&fun, listeningPort(8080));//5005); 3239 return; 3240 } else 3241 version(embedded_httpd_processes) { 3242 import core.sys.posix.unistd; 3243 import core.sys.posix.sys.socket; 3244 import core.sys.posix.netinet.in_; 3245 //import std.c.linux.socket; 3246 3247 int sock = socket(AF_INET, SOCK_STREAM, 0); 3248 if(sock == -1) 3249 throw new Exception("socket"); 3250 3251 { 3252 sockaddr_in addr; 3253 addr.sin_family = AF_INET; 3254 addr.sin_port = htons(listeningPort(8085)); 3255 auto lh = listeningHost(); 3256 if(lh.length) { 3257 if(inet_pton(AF_INET, lh.toStringz(), &addr.sin_addr.s_addr) != 1) 3258 throw new Exception("bad listening host given, please use an IP address.\nExample: --listening-host 127.0.0.1 means listen only on Localhost.\nExample: --listening-host 0.0.0.0 means listen on all interfaces.\nOr you can pass any other single numeric IPv4 address."); 3259 } else 3260 addr.sin_addr.s_addr = INADDR_ANY; 3261 3262 // HACKISH 3263 int on = 1; 3264 setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &on, on.sizeof); 3265 // end hack 3266 3267 3268 if(bind(sock, cast(sockaddr*) &addr, addr.sizeof) == -1) { 3269 close(sock); 3270 throw new Exception("bind"); 3271 } 3272 3273 // FIXME: if this queue is full, it will just ignore it 3274 // and wait for the client to retransmit it. This is an 3275 // obnoxious timeout condition there. 3276 if(sock.listen(128) == -1) { 3277 close(sock); 3278 throw new Exception("listen"); 3279 } 3280 } 3281 3282 version(embedded_httpd_processes_accept_after_fork) {} else { 3283 int pipeReadFd; 3284 int pipeWriteFd; 3285 3286 { 3287 int[2] pipeFd; 3288 if(socketpair(AF_UNIX, SOCK_DGRAM, 0, pipeFd)) { 3289 import core.stdc.errno; 3290 throw new Exception("pipe failed " ~ to!string(errno)); 3291 } 3292 3293 pipeReadFd = pipeFd[0]; 3294 pipeWriteFd = pipeFd[1]; 3295 } 3296 } 3297 3298 3299 int processCount; 3300 pid_t newPid; 3301 reopen: 3302 while(processCount < processPoolSize) { 3303 newPid = fork(); 3304 if(newPid == 0) { 3305 // start serving on the socket 3306 //ubyte[4096] backingBuffer; 3307 for(;;) { 3308 bool closeConnection; 3309 uint i; 3310 sockaddr addr; 3311 i = addr.sizeof; 3312 version(embedded_httpd_processes_accept_after_fork) { 3313 int s = accept(sock, &addr, &i); 3314 int opt = 1; 3315 import core.sys.posix.netinet.tcp; 3316 // the Cgi class does internal buffering, so disabling this 3317 // helps with latency in many cases... 3318 setsockopt(s, IPPROTO_TCP, TCP_NODELAY, &opt, opt.sizeof); 3319 } else { 3320 int s; 3321 auto readret = read_fd(pipeReadFd, &s, s.sizeof, &s); 3322 if(readret != s.sizeof) { 3323 import core.stdc.errno; 3324 throw new Exception("pipe read failed " ~ to!string(errno)); 3325 } 3326 3327 //writeln("process ", getpid(), " got socket ", s); 3328 } 3329 3330 try { 3331 3332 if(s == -1) 3333 throw new Exception("accept"); 3334 3335 scope(failure) close(s); 3336 //ubyte[__traits(classInstanceSize, BufferedInputRange)] bufferedRangeContainer; 3337 auto ir = new BufferedInputRange(s); 3338 //auto ir = emplace!BufferedInputRange(bufferedRangeContainer, s, backingBuffer); 3339 3340 while(!ir.empty) { 3341 ubyte[__traits(classInstanceSize, CustomCgi)] cgiContainer; 3342 3343 Cgi cgi; 3344 try { 3345 cgi = new CustomCgi(ir, &closeConnection); 3346 cgi._outputFileHandle = s; 3347 // if we have a single process and the browser tries to leave the connection open while concurrently requesting another, it will block everything an deadlock since there's no other server to accept it. By closing after each request in this situation, it tells the browser to serialize for us. 3348 if(processPoolSize <= 1) 3349 closeConnection = true; 3350 //cgi = emplace!CustomCgi(cgiContainer, ir, &closeConnection); 3351 } catch(Throwable t) { 3352 // a construction error is either bad code or bad request; bad request is what it should be since this is bug free :P 3353 // anyway let's kill the connection 3354 version(CRuntime_Musl) { 3355 // LockingTextWriter fails here 3356 // so working around it 3357 auto s = t.toString(); 3358 stderr.rawWrite(s); 3359 stderr.rawWrite("\n"); 3360 } else 3361 stderr.writeln(t.toString()); 3362 sendAll(ir.source, plainHttpError(false, "400 Bad Request", t)); 3363 closeConnection = true; 3364 break; 3365 } 3366 assert(cgi !is null); 3367 scope(exit) 3368 cgi.dispose(); 3369 3370 try { 3371 fun(cgi); 3372 cgi.close(); 3373 } catch(ConnectionException ce) { 3374 closeConnection = true; 3375 } catch(Throwable t) { 3376 // a processing error can be recovered from 3377 version(CRuntime_Musl) { 3378 // LockingTextWriter fails here 3379 // so working around it 3380 auto s = t.toString(); 3381 stderr.rawWrite(s); 3382 } else { 3383 stderr.writeln(t.toString); 3384 } 3385 if(!handleException(cgi, t)) 3386 closeConnection = true; 3387 } 3388 3389 if(closeConnection) { 3390 ir.source.close(); 3391 break; 3392 } else { 3393 if(!ir.empty) 3394 ir.popFront(); // get the next 3395 else if(ir.sourceClosed) { 3396 ir.source.close(); 3397 } 3398 } 3399 } 3400 3401 ir.source.close(); 3402 } catch(Throwable t) { 3403 version(CRuntime_Musl) {} else 3404 debug writeln(t); 3405 // most likely cause is a timeout 3406 } 3407 } 3408 } else { 3409 processCount++; 3410 } 3411 } 3412 3413 // the parent should wait for its children... 3414 if(newPid) { 3415 import core.sys.posix.sys.wait; 3416 3417 version(embedded_httpd_processes_accept_after_fork) {} else { 3418 import core.sys.posix.sys.select; 3419 int[] fdQueue; 3420 while(true) { 3421 // writeln("select call"); 3422 int nfds = pipeWriteFd; 3423 if(sock > pipeWriteFd) 3424 nfds = sock; 3425 nfds += 1; 3426 fd_set read_fds; 3427 fd_set write_fds; 3428 FD_ZERO(&read_fds); 3429 FD_ZERO(&write_fds); 3430 FD_SET(sock, &read_fds); 3431 if(fdQueue.length) 3432 FD_SET(pipeWriteFd, &write_fds); 3433 auto ret = select(nfds, &read_fds, &write_fds, null, null); 3434 if(ret == -1) { 3435 import core.stdc.errno; 3436 if(errno == EINTR) 3437 goto try_wait; 3438 else 3439 throw new Exception("wtf select"); 3440 } 3441 3442 int s = -1; 3443 if(FD_ISSET(sock, &read_fds)) { 3444 uint i; 3445 sockaddr addr; 3446 i = addr.sizeof; 3447 s = accept(sock, &addr, &i); 3448 import core.sys.posix.netinet.tcp; 3449 int opt = 1; 3450 setsockopt(s, IPPROTO_TCP, TCP_NODELAY, &opt, opt.sizeof); 3451 } 3452 3453 if(FD_ISSET(pipeWriteFd, &write_fds)) { 3454 if(s == -1 && fdQueue.length) { 3455 s = fdQueue[0]; 3456 fdQueue = fdQueue[1 .. $]; // FIXME reuse buffer 3457 } 3458 write_fd(pipeWriteFd, &s, s.sizeof, s); 3459 close(s); // we are done with it, let the other process take ownership 3460 } else 3461 fdQueue ~= s; 3462 } 3463 } 3464 3465 try_wait: 3466 3467 int status; 3468 while(-1 != wait(&status)) { 3469 version(CRuntime_Musl) {} else { 3470 import std.stdio; writeln("Process died ", status); 3471 } 3472 processCount--; 3473 goto reopen; 3474 } 3475 close(sock); 3476 } 3477 } else 3478 version(embedded_httpd_threads) { 3479 auto manager = new ListeningConnectionManager(listeningHost(), listeningPort(8085), &doThreadHttpConnection!(CustomCgi, fun)); 3480 manager.listen(); 3481 } else 3482 version(scgi) { 3483 import std.exception; 3484 import al = std.algorithm; 3485 auto manager = new ListeningConnectionManager(listeningHost(), listeningPort(4000), &doThreadScgiConnection!(CustomCgi, fun, maxContentLength)); 3486 manager.listen(); 3487 } else 3488 version(fastcgi) { 3489 // SetHandler fcgid-script 3490 FCGX_Stream* input, output, error; 3491 FCGX_ParamArray env; 3492 3493 3494 3495 const(ubyte)[] getFcgiChunk() { 3496 const(ubyte)[] ret; 3497 while(FCGX_HasSeenEOF(input) != -1) 3498 ret ~= cast(ubyte) FCGX_GetChar(input); 3499 return ret; 3500 } 3501 3502 void writeFcgi(const(ubyte)[] data) { 3503 FCGX_PutStr(data.ptr, data.length, output); 3504 } 3505 3506 void doARequest() { 3507 string[string] fcgienv; 3508 3509 for(auto e = env; e !is null && *e !is null; e++) { 3510 string cur = to!string(*e); 3511 auto idx = cur.indexOf("="); 3512 string name, value; 3513 if(idx == -1) 3514 name = cur; 3515 else { 3516 name = cur[0 .. idx]; 3517 value = cur[idx + 1 .. $]; 3518 } 3519 3520 fcgienv[name] = value; 3521 } 3522 3523 void flushFcgi() { 3524 FCGX_FFlush(output); 3525 } 3526 3527 Cgi cgi; 3528 try { 3529 cgi = new CustomCgi(maxContentLength, fcgienv, &getFcgiChunk, &writeFcgi, &flushFcgi); 3530 } catch(Throwable t) { 3531 FCGX_PutStr(cast(ubyte*) t.msg.ptr, t.msg.length, error); 3532 writeFcgi(cast(const(ubyte)[]) plainHttpError(true, "400 Bad Request", t)); 3533 return; //continue; 3534 } 3535 assert(cgi !is null); 3536 scope(exit) cgi.dispose(); 3537 try { 3538 fun(cgi); 3539 cgi.close(); 3540 } catch(Throwable t) { 3541 // log it to the error stream 3542 FCGX_PutStr(cast(ubyte*) t.msg.ptr, t.msg.length, error); 3543 // handle it for the user, if we can 3544 if(!handleException(cgi, t)) 3545 return; // continue; 3546 } 3547 } 3548 3549 auto lp = listeningPort(0); 3550 FCGX_Request request; 3551 if(lp) { 3552 // if a listening port was specified on the command line, we want to spawn ourself 3553 // (needed for nginx without spawn-fcgi, e.g. on Windows) 3554 FCGX_Init(); 3555 auto sock = FCGX_OpenSocket(toStringz(listeningHost() ~ ":" ~ to!string(lp)), 12); 3556 if(sock < 0) 3557 throw new Exception("Couldn't listen on the port"); 3558 FCGX_InitRequest(&request, sock, 0); 3559 while(FCGX_Accept_r(&request) >= 0) { 3560 input = request.inStream; 3561 output = request.outStream; 3562 error = request.errStream; 3563 env = request.envp; 3564 doARequest(); 3565 } 3566 } else { 3567 // otherwise, assume the httpd is doing it (the case for Apache, IIS, and Lighttpd) 3568 // using the version with a global variable since we are separate processes anyway 3569 while(FCGX_Accept(&input, &output, &error, &env) >= 0) { 3570 doARequest(); 3571 } 3572 } 3573 } else { 3574 // standard CGI is the default version 3575 Cgi cgi; 3576 try { 3577 cgi = new CustomCgi(maxContentLength); 3578 version(Posix) 3579 cgi._outputFileHandle = 1; // stdout 3580 else version(Windows) 3581 cgi._outputFileHandle = GetStdHandle(STD_OUTPUT_HANDLE); 3582 else static assert(0); 3583 } catch(Throwable t) { 3584 version(CRuntime_Musl) { 3585 // LockingTextWriter fails here 3586 // so working around it 3587 auto s = t.toString(); 3588 stderr.rawWrite(s); 3589 stdout.rawWrite(plainHttpError(true, "400 Bad Request", t)); 3590 } else { 3591 stderr.writeln(t.msg); 3592 // the real http server will probably handle this; 3593 // most likely, this is a bug in Cgi. But, oh well. 3594 stdout.write(plainHttpError(true, "400 Bad Request", t)); 3595 } 3596 return; 3597 } 3598 assert(cgi !is null); 3599 scope(exit) cgi.dispose(); 3600 3601 try { 3602 fun(cgi); 3603 cgi.close(); 3604 } catch (Throwable t) { 3605 version(CRuntime_Musl) { 3606 // LockingTextWriter fails here 3607 // so working around it 3608 auto s = t.msg; 3609 stderr.rawWrite(s); 3610 } else { 3611 stderr.writeln(t.msg); 3612 } 3613 if(!handleException(cgi, t)) 3614 return; 3615 } 3616 } 3617 } 3618 3619 version(embedded_httpd_threads) 3620 void doThreadHttpConnection(CustomCgi, alias fun)(Socket connection) { 3621 scope(failure) { 3622 // catch all for other errors 3623 sendAll(connection, plainHttpError(false, "500 Internal Server Error", null)); 3624 connection.close(); 3625 } 3626 3627 bool closeConnection; 3628 auto ir = new BufferedInputRange(connection); 3629 3630 while(!ir.empty) { 3631 3632 if(ir.view.length == 0) { 3633 ir.popFront(); 3634 if(ir.sourceClosed) { 3635 connection.close(); 3636 break; 3637 } 3638 } 3639 3640 Cgi cgi; 3641 try { 3642 cgi = new CustomCgi(ir, &closeConnection); 3643 cgi._outputFileHandle = connection.handle; 3644 } catch(ConnectionException ce) { 3645 // broken pipe or something, just abort the connection 3646 closeConnection = true; 3647 break; 3648 } catch(Throwable t) { 3649 // a construction error is either bad code or bad request; bad request is what it should be since this is bug free :P 3650 // anyway let's kill the connection 3651 version(CRuntime_Musl) { 3652 stderr.rawWrite(t.toString()); 3653 stderr.rawWrite("\n"); 3654 } else { 3655 stderr.writeln(t.toString()); 3656 } 3657 sendAll(connection, plainHttpError(false, "400 Bad Request", t)); 3658 closeConnection = true; 3659 break; 3660 } 3661 assert(cgi !is null); 3662 scope(exit) 3663 cgi.dispose(); 3664 3665 try { 3666 fun(cgi); 3667 cgi.close(); 3668 } catch(ConnectionException ce) { 3669 // broken pipe or something, just abort the connection 3670 closeConnection = true; 3671 } catch(Throwable t) { 3672 // a processing error can be recovered from 3673 version(CRuntime_Musl) {} else 3674 stderr.writeln(t.toString); 3675 if(!handleException(cgi, t)) 3676 closeConnection = true; 3677 } 3678 3679 if(closeConnection) { 3680 connection.close(); 3681 break; 3682 } else { 3683 if(ir.front.length) { 3684 ir.popFront(); // we can't just discard the buffer, so get the next bit and keep chugging along 3685 } else if(ir.sourceClosed) { 3686 ir.source.close(); 3687 } else { 3688 continue; 3689 // break; // this was for a keepalive experiment 3690 } 3691 } 3692 } 3693 3694 if(closeConnection) 3695 connection.close(); 3696 3697 // I am otherwise NOT closing it here because the parent thread might still be able to make use of the keep-alive connection! 3698 } 3699 3700 version(scgi) 3701 void doThreadScgiConnection(CustomCgi, alias fun, long maxContentLength)(Socket connection) { 3702 // and now we can buffer 3703 scope(failure) 3704 connection.close(); 3705 3706 import al = std.algorithm; 3707 3708 size_t size; 3709 3710 string[string] headers; 3711 3712 auto range = new BufferedInputRange(connection); 3713 more_data: 3714 auto chunk = range.front(); 3715 // waiting for colon for header length 3716 auto idx = indexOf(cast(string) chunk, ':'); 3717 if(idx == -1) { 3718 try { 3719 range.popFront(); 3720 } catch(Exception e) { 3721 // it is just closed, no big deal 3722 connection.close(); 3723 return; 3724 } 3725 goto more_data; 3726 } 3727 3728 size = to!size_t(cast(string) chunk[0 .. idx]); 3729 chunk = range.consume(idx + 1); 3730 // reading headers 3731 if(chunk.length < size) 3732 range.popFront(0, size + 1); 3733 // we are now guaranteed to have enough 3734 chunk = range.front(); 3735 assert(chunk.length > size); 3736 3737 idx = 0; 3738 string key; 3739 string value; 3740 foreach(part; al.splitter(chunk, '\0')) { 3741 if(idx & 1) { // odd is value 3742 value = cast(string)(part.idup); 3743 headers[key] = value; // commit 3744 } else 3745 key = cast(string)(part.idup); 3746 idx++; 3747 } 3748 3749 enforce(chunk[size] == ','); // the terminator 3750 3751 range.consume(size + 1); 3752 // reading data 3753 // this will be done by Cgi 3754 3755 const(ubyte)[] getScgiChunk() { 3756 // we are already primed 3757 auto data = range.front(); 3758 if(data.length == 0 && !range.sourceClosed) { 3759 range.popFront(0); 3760 data = range.front(); 3761 } else if (range.sourceClosed) 3762 range.source.close(); 3763 3764 return data; 3765 } 3766 3767 void writeScgi(const(ubyte)[] data) { 3768 sendAll(connection, data); 3769 } 3770 3771 void flushScgi() { 3772 // I don't *think* I have to do anything.... 3773 } 3774 3775 Cgi cgi; 3776 try { 3777 cgi = new CustomCgi(maxContentLength, headers, &getScgiChunk, &writeScgi, &flushScgi); 3778 cgi._outputFileHandle = connection.handle; 3779 } catch(Throwable t) { 3780 sendAll(connection, plainHttpError(true, "400 Bad Request", t)); 3781 connection.close(); 3782 return; // this connection is dead 3783 } 3784 assert(cgi !is null); 3785 scope(exit) cgi.dispose(); 3786 try { 3787 fun(cgi); 3788 cgi.close(); 3789 connection.close(); 3790 } catch(Throwable t) { 3791 // no std err 3792 if(!handleException(cgi, t)) { 3793 connection.close(); 3794 return; 3795 } 3796 } 3797 } 3798 3799 string printDate(DateTime date) { 3800 return format( 3801 "%.3s, %02d %.3s %d %02d:%02d:%02d GMT", // could be UTC too 3802 to!string(date.dayOfWeek).capitalize, 3803 date.day, 3804 to!string(date.month).capitalize, 3805 date.year, 3806 date.hour, 3807 date.minute, 3808 date.second); 3809 } 3810 3811 3812 // Referencing this gigantic typeid seems to remind the compiler 3813 // to actually put the symbol in the object file. I guess the immutable 3814 // assoc array array isn't actually included in druntime 3815 void hackAroundLinkerError() { 3816 stdout.rawWrite(typeid(const(immutable(char)[][])[immutable(char)[]]).toString()); 3817 stdout.rawWrite(typeid(immutable(char)[][][immutable(char)[]]).toString()); 3818 stdout.rawWrite(typeid(Cgi.UploadedFile[immutable(char)[]]).toString()); 3819 stdout.rawWrite(typeid(Cgi.UploadedFile[][immutable(char)[]]).toString()); 3820 stdout.rawWrite(typeid(immutable(Cgi.UploadedFile)[immutable(char)[]]).toString()); 3821 stdout.rawWrite(typeid(immutable(Cgi.UploadedFile[])[immutable(char)[]]).toString()); 3822 stdout.rawWrite(typeid(immutable(char[])[immutable(char)[]]).toString()); 3823 // this is getting kinda ridiculous btw. Moving assoc arrays 3824 // to the library is the pain that keeps on coming. 3825 3826 // eh this broke the build on the work server 3827 // stdout.rawWrite(typeid(immutable(char)[][immutable(string[])])); 3828 stdout.rawWrite(typeid(immutable(string[])[immutable(char)[]]).toString()); 3829 } 3830 3831 3832 3833 3834 3835 version(fastcgi) { 3836 pragma(lib, "fcgi"); 3837 3838 static if(size_t.sizeof == 8) // 64 bit 3839 alias long c_int; 3840 else 3841 alias int c_int; 3842 3843 extern(C) { 3844 struct FCGX_Stream { 3845 ubyte* rdNext; 3846 ubyte* wrNext; 3847 ubyte* stop; 3848 ubyte* stopUnget; 3849 c_int isReader; 3850 c_int isClosed; 3851 c_int wasFCloseCalled; 3852 c_int FCGI_errno; 3853 void* function(FCGX_Stream* stream) fillBuffProc; 3854 void* function(FCGX_Stream* stream, c_int doClose) emptyBuffProc; 3855 void* data; 3856 } 3857 3858 // note: this is meant to be opaque, so don't access it directly 3859 struct FCGX_Request { 3860 int requestId; 3861 int role; 3862 FCGX_Stream* inStream; 3863 FCGX_Stream* outStream; 3864 FCGX_Stream* errStream; 3865 char** envp; 3866 void* paramsPtr; 3867 int ipcFd; 3868 int isBeginProcessed; 3869 int keepConnection; 3870 int appStatus; 3871 int nWriters; 3872 int flags; 3873 int listen_sock; 3874 } 3875 3876 int FCGX_InitRequest(FCGX_Request *request, int sock, int flags); 3877 void FCGX_Init(); 3878 3879 int FCGX_Accept_r(FCGX_Request *request); 3880 3881 3882 alias char** FCGX_ParamArray; 3883 3884 c_int FCGX_Accept(FCGX_Stream** stdin, FCGX_Stream** stdout, FCGX_Stream** stderr, FCGX_ParamArray* envp); 3885 c_int FCGX_GetChar(FCGX_Stream* stream); 3886 c_int FCGX_PutStr(const ubyte* str, c_int n, FCGX_Stream* stream); 3887 int FCGX_HasSeenEOF(FCGX_Stream* stream); 3888 c_int FCGX_FFlush(FCGX_Stream *stream); 3889 3890 int FCGX_OpenSocket(in char*, int); 3891 } 3892 } 3893 3894 3895 /* This might go int a separate module eventually. It is a network input helper class. */ 3896 3897 import std.socket; 3898 3899 // it is a class primarily for reference semantics 3900 // I might change this interface 3901 /// This is NOT ACTUALLY an input range! It is too different. Historical mistake kinda. 3902 class BufferedInputRange { 3903 version(Posix) 3904 this(int source, ubyte[] buffer = null) { 3905 this(new Socket(cast(socket_t) source, AddressFamily.INET), buffer); 3906 } 3907 3908 this(Socket source, ubyte[] buffer = null) { 3909 // if they connect but never send stuff to us, we don't want it wasting the process 3910 // so setting a time out 3911 source.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVTIMEO, dur!"seconds"(3)); 3912 this.source = source; 3913 if(buffer is null) { 3914 underlyingBuffer = new ubyte[4096]; 3915 allowGrowth = true; 3916 } else { 3917 underlyingBuffer = buffer; 3918 } 3919 3920 assert(underlyingBuffer.length); 3921 3922 // we assume view.ptr is always inside underlyingBuffer 3923 view = underlyingBuffer[0 .. 0]; 3924 3925 popFront(); // prime 3926 } 3927 3928 /** 3929 A slight difference from regular ranges is you can give it the maximum 3930 number of bytes to consume. 3931 3932 IMPORTANT NOTE: the default is to consume nothing, so if you don't call 3933 consume() yourself and use a regular foreach, it will infinitely loop! 3934 3935 The default is to do what a normal range does, and consume the whole buffer 3936 and wait for additional input. 3937 3938 You can also specify 0, to append to the buffer, or any other number 3939 to remove the front n bytes and wait for more. 3940 */ 3941 void popFront(size_t maxBytesToConsume = 0 /*size_t.max*/, size_t minBytesToSettleFor = 0, bool skipConsume = false) { 3942 if(sourceClosed) 3943 throw new Exception("can't get any more data from a closed source"); 3944 if(!skipConsume) 3945 consume(maxBytesToConsume); 3946 3947 // we might have to grow the buffer 3948 if(minBytesToSettleFor > underlyingBuffer.length || view.length == underlyingBuffer.length) { 3949 if(allowGrowth) { 3950 auto viewStart = view.ptr - underlyingBuffer.ptr; 3951 size_t growth = 4096; 3952 // make sure we have enough for what we're being asked for 3953 if(minBytesToSettleFor - underlyingBuffer.length > growth) 3954 growth = minBytesToSettleFor - underlyingBuffer.length; 3955 underlyingBuffer.length += growth; 3956 view = underlyingBuffer[viewStart .. view.length]; 3957 } else 3958 throw new Exception("No room left in the buffer"); 3959 } 3960 3961 do { 3962 auto freeSpace = underlyingBuffer[underlyingBuffer.ptr - view.ptr + view.length .. $]; 3963 try_again: 3964 auto ret = source.receive(freeSpace); 3965 if(ret == Socket.ERROR) { 3966 if(wouldHaveBlocked()) { 3967 // gonna treat a timeout here as a close 3968 sourceClosed = true; 3969 return; 3970 } 3971 version(Posix) { 3972 import core.stdc.errno; 3973 if(errno == EINTR || errno == EAGAIN) { 3974 goto try_again; 3975 } 3976 } 3977 throw new Exception(lastSocketError); // FIXME 3978 } 3979 if(ret == 0) { 3980 sourceClosed = true; 3981 return; 3982 } 3983 3984 view = underlyingBuffer[view.ptr - underlyingBuffer.ptr .. view.length + ret]; 3985 } while(view.length < minBytesToSettleFor); 3986 } 3987 3988 /// Removes n bytes from the front of the buffer, and returns the new buffer slice. 3989 /// You might want to idup the data you are consuming if you store it, since it may 3990 /// be overwritten on the new popFront. 3991 /// 3992 /// You do not need to call this if you always want to wait for more data when you 3993 /// consume some. 3994 ubyte[] consume(size_t bytes) { 3995 view = view[bytes > $ ? $ : bytes .. $]; 3996 if(view.length == 0) { 3997 view = underlyingBuffer[0 .. 0]; // go ahead and reuse the beginning 3998 /* 3999 writeln("HERE"); 4000 popFront(0, 0, true); // try to load more if we can, checks if the source is closed 4001 writeln(cast(string)front); 4002 writeln("DONE"); 4003 */ 4004 } 4005 return front; 4006 } 4007 4008 bool empty() { 4009 return sourceClosed && view.length == 0; 4010 } 4011 4012 ubyte[] front() { 4013 return view; 4014 } 4015 4016 invariant() { 4017 assert(view.ptr >= underlyingBuffer.ptr); 4018 // it should never be equal, since if that happens view ought to be empty, and thus reusing the buffer 4019 assert(view.ptr < underlyingBuffer.ptr + underlyingBuffer.length); 4020 } 4021 4022 ubyte[] underlyingBuffer; 4023 bool allowGrowth; 4024 ubyte[] view; 4025 Socket source; 4026 bool sourceClosed; 4027 } 4028 4029 import core.sync.semaphore; 4030 import core.atomic; 4031 4032 /** 4033 To use this thing: 4034 4035 void handler(Socket s) { do something... } 4036 auto manager = new ListeningConnectionManager("127.0.0.1", 80, &handler); 4037 manager.listen(); 4038 4039 I suggest you use BufferedInputRange(connection) to handle the input. As a packet 4040 comes in, you will get control. You can just continue; though to fetch more. 4041 4042 4043 FIXME: should I offer an event based async thing like netman did too? Yeah, probably. 4044 */ 4045 class ListeningConnectionManager { 4046 Semaphore semaphore; 4047 Socket[256] queue; 4048 shared(ubyte) nextIndexFront; 4049 ubyte nextIndexBack; 4050 shared(int) queueLength; 4051 4052 void listen() { 4053 running = true; 4054 shared(int) loopBroken; 4055 4056 version(cgi_no_threads) { 4057 // NEVER USE THIS 4058 // it exists only for debugging and other special occasions 4059 4060 // the thread mode is faster and less likely to stall the whole 4061 // thing when a request is slow 4062 while(!loopBroken && running) { 4063 auto sn = listener.accept(); 4064 try { 4065 handler(sn); 4066 } catch(Exception e) { 4067 // if a connection goes wrong, we want to just say no, but try to carry on unless it is an Error of some sort (in which case, we'll die. You might want an external helper program to revive the server when it dies) 4068 sn.close(); 4069 } 4070 } 4071 } else { 4072 semaphore = new Semaphore(); 4073 4074 ConnectionThread[16] threads; 4075 foreach(i, ref thread; threads) { 4076 thread = new ConnectionThread(this, handler, cast(int) i); 4077 thread.start(); 4078 } 4079 4080 /+ 4081 version(linux) { 4082 import core.sys.linux.epoll; 4083 epoll_fd = epoll_create1(EPOLL_CLOEXEC); 4084 if(epoll_fd == -1) 4085 throw new Exception("epoll_create1 " ~ to!string(errno)); 4086 scope(exit) { 4087 import core.sys.posix.unistd; 4088 close(epoll_fd); 4089 } 4090 4091 epoll_event[64] events; 4092 4093 epoll_event ev; 4094 ev.events = EPOLLIN; 4095 ev.data.fd = listener.handle; 4096 if(epoll_ctl(epoll_fd, EPOLL_CTL_ADD, listener.handle, &ev) == -1) 4097 throw new Exception("epoll_ctl " ~ to!string(errno)); 4098 } 4099 +/ 4100 4101 while(!loopBroken && running) { 4102 Socket sn; 4103 4104 void accept_new_connection() { 4105 sn = listener.accept(); 4106 if(tcp) { 4107 // disable Nagle's algorithm to avoid a 40ms delay when we send/recv 4108 // on the socket because we do some buffering internally. I think this helps, 4109 // certainly does for small requests, and I think it does for larger ones too 4110 sn.setOption(SocketOptionLevel.TCP, SocketOption.TCP_NODELAY, 1); 4111 4112 sn.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVTIMEO, dur!"seconds"(10)); 4113 } 4114 } 4115 4116 void existing_connection_new_data() { 4117 // wait until a slot opens up 4118 //int waited = 0; 4119 while(queueLength >= queue.length) { 4120 Thread.sleep(1.msecs); 4121 //waited ++; 4122 } 4123 //if(waited) {import std.stdio; writeln(waited);} 4124 synchronized(this) { 4125 queue[nextIndexBack] = sn; 4126 nextIndexBack++; 4127 atomicOp!"+="(queueLength, 1); 4128 } 4129 semaphore.notify(); 4130 } 4131 4132 bool crash_check() { 4133 bool hasAnyRunning; 4134 foreach(thread; threads) { 4135 if(!thread.isRunning) { 4136 thread.join(); 4137 } else hasAnyRunning = true; 4138 } 4139 4140 return (!hasAnyRunning); 4141 } 4142 4143 4144 /+ 4145 version(linux) { 4146 auto nfds = epoll_wait(epoll_fd, events.ptr, events.length, -1); 4147 if(nfds == -1) { 4148 if(errno == EINTR) 4149 continue; 4150 throw new Exception("epoll_wait " ~ to!string(errno)); 4151 } 4152 4153 foreach(idx; 0 .. nfds) { 4154 auto flags = events[idx].events; 4155 auto fd = events[idx].data.fd; 4156 4157 if(fd == listener.handle) { 4158 accept_new_connection(); 4159 existing_connection_new_data(); 4160 } else { 4161 if(flags & (EPOLLHUP | EPOLLERR | EPOLLRDHUP)) { 4162 import core.sys.posix.unistd; 4163 close(fd); 4164 } else { 4165 sn = new Socket(cast(socket_t) fd, tcp ? AddressFamily.INET : AddressFamily.UNIX); 4166 import std.stdio; writeln("existing_connection_new_data"); 4167 existing_connection_new_data(); 4168 } 4169 } 4170 } 4171 } else { 4172 +/ 4173 accept_new_connection(); 4174 existing_connection_new_data(); 4175 //} 4176 4177 if(crash_check()) 4178 break; 4179 } 4180 4181 // FIXME: i typically stop this with ctrl+c which never 4182 // actually gets here. i need to do a sigint handler. 4183 if(cleanup) 4184 cleanup(); 4185 } 4186 } 4187 4188 //version(linux) 4189 //int epoll_fd; 4190 4191 bool tcp; 4192 void delegate() cleanup; 4193 4194 this(string host, ushort port, void function(Socket) handler) { 4195 this.handler = handler; 4196 4197 if(host.startsWith("unix:")) { 4198 version(Posix) { 4199 listener = new Socket(AddressFamily.UNIX, SocketType.STREAM); 4200 string filename = host["unix:".length .. $].idup; 4201 listener.bind(new UnixAddress(filename)); 4202 cleanup = delegate() { 4203 import std.file; 4204 remove(filename); 4205 }; 4206 tcp = false; 4207 } else { 4208 throw new Exception("unix sockets not supported on this system"); 4209 } 4210 } else if(host.startsWith("abstract:")) { 4211 version(linux) { 4212 listener = new Socket(AddressFamily.UNIX, SocketType.STREAM); 4213 string filename = "\0" ~ host["abstract:".length .. $]; 4214 listener.bind(new UnixAddress(filename)); 4215 tcp = false; 4216 } else { 4217 throw new Exception("abstract unix sockets not supported on this system"); 4218 } 4219 } else { 4220 listener = new TcpSocket(); 4221 listener.setOption(SocketOptionLevel.SOCKET, SocketOption.REUSEADDR, true); 4222 listener.bind(host.length ? parseAddress(host, port) : new InternetAddress(port)); 4223 tcp = true; 4224 } 4225 4226 Thread.getThis.priority = Thread.PRIORITY_MAX; 4227 listener.listen(128); 4228 } 4229 4230 Socket listener; 4231 void function(Socket) handler; 4232 4233 bool running; 4234 void quit() { 4235 running = false; 4236 } 4237 } 4238 4239 // helper function to send a lot to a socket. Since this blocks for the buffer (possibly several times), you should probably call it in a separate thread or something. 4240 void sendAll(Socket s, const(void)[] data, string file = __FILE__, size_t line = __LINE__) { 4241 if(data.length == 0) return; 4242 ptrdiff_t amount; 4243 do { 4244 amount = s.send(data); 4245 if(amount == Socket.ERROR) 4246 throw new ConnectionException(s, lastSocketError, file, line); 4247 assert(amount > 0); 4248 data = data[amount .. $]; 4249 } while(data.length); 4250 } 4251 4252 class ConnectionException : Exception { 4253 Socket socket; 4254 this(Socket s, string msg, string file = __FILE__, size_t line = __LINE__) { 4255 this.socket = s; 4256 super(msg, file, line); 4257 } 4258 } 4259 4260 alias void function(Socket) CMT; 4261 4262 import core.thread; 4263 /+ 4264 cgi.d now uses a hybrid of event i/o and threads at the top level. 4265 4266 Top level thread is responsible for accepting sockets and selecting on them. 4267 4268 It then indicates to a child that a request is pending, and any random worker 4269 thread that is free handles it. It goes into blocking mode and handles that 4270 http request to completion. 4271 4272 At that point, it goes back into the waiting queue. 4273 4274 4275 This concept is only implemented on Linux. On all other systems, it still 4276 uses the worker threads and semaphores (which is perfectly fine for a lot of 4277 things! Just having a great number of keep-alive connections will break that.) 4278 4279 4280 So the algorithm is: 4281 4282 select(accept, event, pending) 4283 if accept -> send socket to free thread, if any. if not, add socket to queue 4284 if event -> send the signaling thread a socket from the queue, if not, mark it free 4285 - event might block until it can be *written* to. it is a fifo sending socket fds! 4286 4287 A worker only does one http request at a time, then signals its availability back to the boss. 4288 4289 The socket the worker was just doing should be added to the one-off epoll read. If it is closed, 4290 great, we can get rid of it. Otherwise, it is considered `pending`. The *kernel* manages that; the 4291 actual FD will not be kept out here. 4292 4293 So: 4294 queue = sockets we know are ready to read now, but no worker thread is available 4295 idle list = worker threads not doing anything else. they signal back and forth 4296 4297 the workers all read off the event fd. This is the semaphore wait 4298 4299 the boss waits on accept or other sockets read events (one off! and level triggered). If anything happens wrt ready read, 4300 it puts it in the queue and writes to the event fd. 4301 4302 The child could put the socket back in the epoll thing itself. 4303 4304 The child needs to be able to gracefully handle being given a socket that just closed with no work. 4305 +/ 4306 class ConnectionThread : Thread { 4307 this(ListeningConnectionManager lcm, CMT dg, int myThreadNumber) { 4308 this.lcm = lcm; 4309 this.dg = dg; 4310 this.myThreadNumber = myThreadNumber; 4311 super(&run); 4312 } 4313 4314 void run() { 4315 while(true) { 4316 // so if there's a bunch of idle keep-alive connections, it can 4317 // consume all the worker threads... just sitting there. 4318 lcm.semaphore.wait(); 4319 Socket socket; 4320 synchronized(lcm) { 4321 auto idx = lcm.nextIndexFront; 4322 socket = lcm.queue[idx]; 4323 lcm.queue[idx] = null; 4324 atomicOp!"+="(lcm.nextIndexFront, 1); 4325 atomicOp!"-="(lcm.queueLength, 1); 4326 } 4327 try { 4328 //import std.stdio; writeln(myThreadNumber, " taking it"); 4329 dg(socket); 4330 /+ 4331 if(socket.isAlive) { 4332 // process it more later 4333 version(linux) { 4334 import core.sys.linux.epoll; 4335 epoll_event ev; 4336 ev.events = EPOLLIN | EPOLLONESHOT | EPOLLET; 4337 ev.data.fd = socket.handle; 4338 import std.stdio; writeln("adding"); 4339 if(epoll_ctl(lcm.epoll_fd, EPOLL_CTL_ADD, socket.handle, &ev) == -1) { 4340 if(errno == EEXIST) { 4341 ev.events = EPOLLIN | EPOLLONESHOT | EPOLLET; 4342 ev.data.fd = socket.handle; 4343 if(epoll_ctl(lcm.epoll_fd, EPOLL_CTL_MOD, socket.handle, &ev) == -1) 4344 throw new Exception("epoll_ctl " ~ to!string(errno)); 4345 } else 4346 throw new Exception("epoll_ctl " ~ to!string(errno)); 4347 } 4348 //import std.stdio; writeln("keep alive"); 4349 // writing to this private member is to prevent the GC from closing my precious socket when I'm trying to use it later 4350 __traits(getMember, socket, "sock") = cast(socket_t) -1; 4351 } else { 4352 continue; // hope it times out in a reasonable amount of time... 4353 } 4354 } 4355 +/ 4356 } catch(Throwable e) { 4357 import std.stdio; stderr.rawWrite(e.toString); stderr.rawWrite("\n"); 4358 socket.close(); 4359 } 4360 } 4361 } 4362 4363 ListeningConnectionManager lcm; 4364 CMT dg; 4365 int myThreadNumber; 4366 } 4367 4368 /* Done with network helper */ 4369 4370 /* Helpers for doing temporary files. Used both here and in web.d */ 4371 4372 version(Windows) { 4373 import core.sys.windows.windows; 4374 extern(Windows) DWORD GetTempPathW(DWORD, LPWSTR); 4375 alias GetTempPathW GetTempPath; 4376 } 4377 4378 version(Posix) { 4379 static import linux = core.sys.posix.unistd; 4380 } 4381 4382 string getTempDirectory() { 4383 string path; 4384 version(Windows) { 4385 wchar[1024] buffer; 4386 auto len = GetTempPath(1024, buffer.ptr); 4387 if(len == 0) 4388 throw new Exception("couldn't find a temporary path"); 4389 4390 auto b = buffer[0 .. len]; 4391 4392 path = to!string(b); 4393 } else 4394 path = "/tmp/"; 4395 4396 return path; 4397 } 4398 4399 4400 // I like std.date. These functions help keep my old code and data working with phobos changing. 4401 4402 long sysTimeToDTime(in SysTime sysTime) { 4403 return convert!("hnsecs", "msecs")(sysTime.stdTime - 621355968000000000L); 4404 } 4405 4406 long dateTimeToDTime(in DateTime dt) { 4407 return sysTimeToDTime(cast(SysTime) dt); 4408 } 4409 4410 long getUtcTime() { // renamed primarily to avoid conflict with std.date itself 4411 return sysTimeToDTime(Clock.currTime(UTC())); 4412 } 4413 4414 // NOTE: new SimpleTimeZone(minutes); can perhaps work with the getTimezoneOffset() JS trick 4415 SysTime dTimeToSysTime(long dTime, immutable TimeZone tz = null) { 4416 immutable hnsecs = convert!("msecs", "hnsecs")(dTime) + 621355968000000000L; 4417 return SysTime(hnsecs, tz); 4418 } 4419 4420 4421 4422 // this is a helper to read HTTP transfer-encoding: chunked responses 4423 immutable(ubyte[]) dechunk(BufferedInputRange ir) { 4424 immutable(ubyte)[] ret; 4425 4426 another_chunk: 4427 // If here, we are at the beginning of a chunk. 4428 auto a = ir.front(); 4429 int chunkSize; 4430 int loc = locationOf(a, "\r\n"); 4431 while(loc == -1) { 4432 ir.popFront(); 4433 a = ir.front(); 4434 loc = locationOf(a, "\r\n"); 4435 } 4436 4437 string hex; 4438 hex = ""; 4439 for(int i = 0; i < loc; i++) { 4440 char c = a[i]; 4441 if(c >= 'A' && c <= 'Z') 4442 c += 0x20; 4443 if((c >= '0' && c <= '9') || (c >= 'a' && c <= 'z')) { 4444 hex ~= c; 4445 } else { 4446 break; 4447 } 4448 } 4449 4450 assert(hex.length); 4451 4452 int power = 1; 4453 int size = 0; 4454 foreach(cc1; retro(hex)) { 4455 dchar cc = cc1; 4456 if(cc >= 'a' && cc <= 'z') 4457 cc -= 0x20; 4458 int val = 0; 4459 if(cc >= '0' && cc <= '9') 4460 val = cc - '0'; 4461 else 4462 val = cc - 'A' + 10; 4463 4464 size += power * val; 4465 power *= 16; 4466 } 4467 4468 chunkSize = size; 4469 assert(size >= 0); 4470 4471 if(loc + 2 > a.length) { 4472 ir.popFront(0, a.length + loc + 2); 4473 a = ir.front(); 4474 } 4475 4476 a = ir.consume(loc + 2); 4477 4478 if(chunkSize == 0) { // we're done with the response 4479 // if we got here, will change must be true.... 4480 more_footers: 4481 loc = locationOf(a, "\r\n"); 4482 if(loc == -1) { 4483 ir.popFront(); 4484 a = ir.front; 4485 goto more_footers; 4486 } else { 4487 assert(loc == 0); 4488 ir.consume(loc + 2); 4489 goto finish; 4490 } 4491 } else { 4492 // if we got here, will change must be true.... 4493 if(a.length < chunkSize + 2) { 4494 ir.popFront(0, chunkSize + 2); 4495 a = ir.front(); 4496 } 4497 4498 ret ~= (a[0..chunkSize]); 4499 4500 if(!(a.length > chunkSize + 2)) { 4501 ir.popFront(0, chunkSize + 2); 4502 a = ir.front(); 4503 } 4504 assert(a[chunkSize] == 13); 4505 assert(a[chunkSize+1] == 10); 4506 a = ir.consume(chunkSize + 2); 4507 chunkSize = 0; 4508 goto another_chunk; 4509 } 4510 4511 finish: 4512 return ret; 4513 } 4514 4515 // I want to be able to get data from multiple sources the same way... 4516 interface ByChunkRange { 4517 bool empty(); 4518 void popFront(); 4519 const(ubyte)[] front(); 4520 } 4521 4522 ByChunkRange byChunk(const(ubyte)[] data) { 4523 return new class ByChunkRange { 4524 override bool empty() { 4525 return !data.length; 4526 } 4527 4528 override void popFront() { 4529 if(data.length > 4096) 4530 data = data[4096 .. $]; 4531 else 4532 data = null; 4533 } 4534 4535 override const(ubyte)[] front() { 4536 return data[0 .. $ > 4096 ? 4096 : $]; 4537 } 4538 }; 4539 } 4540 4541 ByChunkRange byChunk(BufferedInputRange ir, size_t atMost) { 4542 const(ubyte)[] f; 4543 4544 f = ir.front; 4545 if(f.length > atMost) 4546 f = f[0 .. atMost]; 4547 4548 return new class ByChunkRange { 4549 override bool empty() { 4550 return atMost == 0; 4551 } 4552 4553 override const(ubyte)[] front() { 4554 return f; 4555 } 4556 4557 override void popFront() { 4558 ir.consume(f.length); 4559 atMost -= f.length; 4560 auto a = ir.front(); 4561 4562 if(a.length <= atMost) { 4563 f = a; 4564 atMost -= a.length; 4565 a = ir.consume(a.length); 4566 if(atMost != 0) 4567 ir.popFront(); 4568 if(f.length == 0) { 4569 f = ir.front(); 4570 } 4571 } else { 4572 // we actually have *more* here than we need.... 4573 f = a[0..atMost]; 4574 atMost = 0; 4575 ir.consume(atMost); 4576 } 4577 } 4578 }; 4579 } 4580 4581 version(cgi_with_websocket) { 4582 // http://tools.ietf.org/html/rfc6455 4583 4584 /** 4585 WEBSOCKET SUPPORT: 4586 4587 Full example: 4588 --- 4589 import arsd.cgi; 4590 4591 void websocketEcho(Cgi cgi) { 4592 if(cgi.websocketRequested()) { 4593 if(cgi.origin != "http://arsdnet.net") 4594 throw new Exception("bad origin"); 4595 auto websocket = cgi.acceptWebsocket(); 4596 4597 websocket.send("hello"); 4598 websocket.send(" world!"); 4599 4600 auto msg = websocket.recv(); 4601 while(msg.opcode != WebSocketOpcode.close) { 4602 if(msg.opcode == WebSocketOpcode.text) { 4603 websocket.send(msg.textData); 4604 } else if(msg.opcode == WebSocketOpcode.binary) { 4605 websocket.send(msg.data); 4606 } 4607 4608 msg = websocket.recv(); 4609 } 4610 4611 websocket.close(); 4612 } else assert(0, "i want a web socket!"); 4613 } 4614 4615 mixin GenericMain!websocketEcho; 4616 --- 4617 */ 4618 4619 class WebSocket { 4620 Cgi cgi; 4621 4622 private this(Cgi cgi) { 4623 this.cgi = cgi; 4624 } 4625 4626 // returns true if data available, false if it timed out 4627 bool recvAvailable(Duration timeout = dur!"msecs"(0)) { 4628 Socket socket = cgi.idlol.source; 4629 4630 auto check = new SocketSet(); 4631 check.add(socket); 4632 4633 auto got = Socket.select(check, null, null, timeout); 4634 if(got > 0) 4635 return true; 4636 return false; 4637 } 4638 4639 // note: this blocks 4640 WebSocketMessage recv() { 4641 // FIXME: should we automatically handle pings and pongs? 4642 assert(!cgi.idlol.empty()); 4643 cgi.idlol.popFront(0); 4644 4645 WebSocketMessage message; 4646 4647 auto info = cgi.idlol.front(); 4648 4649 // FIXME: read should prolly take the whole range so it can request more if needed 4650 // read should also go ahead and consume the range 4651 message = WebSocketMessage.read(info); 4652 4653 cgi.idlol.consume(info.length); 4654 4655 return message; 4656 } 4657 4658 void send(in char[] text) { 4659 // I cast away const here because I know this msg is private and it doesn't write 4660 // to that buffer unless masking is set... which it isn't, so we're ok. 4661 auto msg = WebSocketMessage.simpleMessage(WebSocketOpcode.text, cast(void[]) text); 4662 msg.send(cgi); 4663 } 4664 4665 void send(in ubyte[] binary) { 4666 // I cast away const here because I know this msg is private and it doesn't write 4667 // to that buffer unless masking is set... which it isn't, so we're ok. 4668 auto msg = WebSocketMessage.simpleMessage(WebSocketOpcode.binary, cast(void[]) binary); 4669 msg.send(cgi); 4670 } 4671 4672 void close() { 4673 auto msg = WebSocketMessage.simpleMessage(WebSocketOpcode.close, null); 4674 msg.send(cgi); 4675 } 4676 4677 void ping() { 4678 auto msg = WebSocketMessage.simpleMessage(WebSocketOpcode.ping, null); 4679 msg.send(cgi); 4680 } 4681 4682 void pong() { 4683 auto msg = WebSocketMessage.simpleMessage(WebSocketOpcode.pong, null); 4684 msg.send(cgi); 4685 } 4686 } 4687 4688 bool websocketRequested(Cgi cgi) { 4689 return 4690 "sec-websocket-key" in cgi.requestHeaders 4691 && 4692 "connection" in cgi.requestHeaders && 4693 cgi.requestHeaders["connection"].asLowerCase().canFind("upgrade") 4694 && 4695 "upgrade" in cgi.requestHeaders && 4696 cgi.requestHeaders["upgrade"].asLowerCase().equal("websocket") 4697 ; 4698 } 4699 4700 WebSocket acceptWebsocket(Cgi cgi) { 4701 assert(!cgi.closed); 4702 assert(!cgi.outputtedResponseData); 4703 cgi.setResponseStatus("101 Web Socket Protocol Handshake"); 4704 cgi.header("Upgrade: WebSocket"); 4705 cgi.header("Connection: upgrade"); 4706 4707 string key = cgi.requestHeaders["sec-websocket-key"]; 4708 key ~= "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"; // the defined guid from the websocket spec 4709 4710 import std.digest.sha; 4711 auto hash = sha1Of(key); 4712 auto accept = Base64.encode(hash); 4713 4714 cgi.header(("Sec-WebSocket-Accept: " ~ accept).idup); 4715 4716 cgi.websocketMode = true; 4717 cgi.write(""); 4718 4719 cgi.flush(); 4720 4721 return new WebSocket(cgi); 4722 } 4723 4724 // FIXME: implement websocket extension frames 4725 // get websocket to work on other modes, not just embedded_httpd 4726 4727 enum WebSocketOpcode : ubyte { 4728 text = 1, 4729 binary = 2, 4730 // 3, 4, 5, 6, 7 RESERVED 4731 close = 8, 4732 ping = 9, 4733 pong = 10, 4734 // 11,12,13,14,15 RESERVED 4735 } 4736 4737 struct WebSocketMessage { 4738 bool fin; 4739 bool rsv1; 4740 bool rsv2; 4741 bool rsv3; 4742 WebSocketOpcode opcode; // 4 bits 4743 bool masked; 4744 ubyte lengthIndicator; // don't set this when building one to send 4745 ulong realLength; // don't use when sending 4746 ubyte[4] maskingKey; // don't set this when sending 4747 ubyte[] data; 4748 4749 static WebSocketMessage simpleMessage(WebSocketOpcode opcode, void[] data) { 4750 WebSocketMessage msg; 4751 msg.fin = true; 4752 msg.opcode = opcode; 4753 msg.data = cast(ubyte[]) data; 4754 4755 return msg; 4756 } 4757 4758 private void send(Cgi cgi) { 4759 ubyte[64] headerScratch; 4760 int headerScratchPos = 0; 4761 4762 realLength = data.length; 4763 4764 { 4765 ubyte b1; 4766 b1 |= cast(ubyte) opcode; 4767 b1 |= rsv3 ? (1 << 4) : 0; 4768 b1 |= rsv2 ? (1 << 5) : 0; 4769 b1 |= rsv1 ? (1 << 6) : 0; 4770 b1 |= fin ? (1 << 7) : 0; 4771 4772 headerScratch[0] = b1; 4773 headerScratchPos++; 4774 } 4775 4776 { 4777 headerScratchPos++; // we'll set header[1] at the end of this 4778 auto rlc = realLength; 4779 ubyte b2; 4780 b2 |= masked ? (1 << 7) : 0; 4781 4782 assert(headerScratchPos == 2); 4783 4784 if(realLength > 65535) { 4785 // use 64 bit length 4786 b2 |= 0x7f; 4787 4788 // FIXME: double check endinaness 4789 foreach(i; 0 .. 8) { 4790 headerScratch[2 + 7 - i] = rlc & 0x0ff; 4791 rlc >>>= 8; 4792 } 4793 4794 headerScratchPos += 8; 4795 } else if(realLength > 127) { 4796 // use 16 bit length 4797 b2 |= 0x7e; 4798 4799 // FIXME: double check endinaness 4800 foreach(i; 0 .. 2) { 4801 headerScratch[2 + 1 - i] = rlc & 0x0ff; 4802 rlc >>>= 8; 4803 } 4804 4805 headerScratchPos += 2; 4806 } else { 4807 // use 7 bit length 4808 b2 |= realLength & 0b_0111_1111; 4809 } 4810 4811 headerScratch[1] = b2; 4812 } 4813 4814 assert(!masked, "masking key not properly implemented"); 4815 if(masked) { 4816 // FIXME: randomize this 4817 headerScratch[headerScratchPos .. headerScratchPos + 4] = maskingKey[]; 4818 headerScratchPos += 4; 4819 4820 // we'll just mask it in place... 4821 int keyIdx = 0; 4822 foreach(i; 0 .. data.length) { 4823 data[i] = data[i] ^ maskingKey[keyIdx]; 4824 if(keyIdx == 3) 4825 keyIdx = 0; 4826 else 4827 keyIdx++; 4828 } 4829 } 4830 4831 //writeln("SENDING ", headerScratch[0 .. headerScratchPos], data); 4832 cgi.write(headerScratch[0 .. headerScratchPos]); 4833 cgi.write(data); 4834 cgi.flush(); 4835 } 4836 4837 static WebSocketMessage read(ubyte[] d) { 4838 WebSocketMessage msg; 4839 assert(d.length >= 2); 4840 4841 ubyte b = d[0]; 4842 4843 msg.opcode = cast(WebSocketOpcode) (b & 0x0f); 4844 b >>= 4; 4845 msg.rsv3 = b & 0x01; 4846 b >>= 1; 4847 msg.rsv2 = b & 0x01; 4848 b >>= 1; 4849 msg.rsv1 = b & 0x01; 4850 b >>= 1; 4851 msg.fin = b & 0x01; 4852 4853 b = d[1]; 4854 msg.masked = (b & 0b1000_0000) ? true : false; 4855 msg.lengthIndicator = b & 0b0111_1111; 4856 4857 d = d[2 .. $]; 4858 4859 if(msg.lengthIndicator == 0x7e) { 4860 // 16 bit length 4861 msg.realLength = 0; 4862 4863 foreach(i; 0 .. 2) { 4864 msg.realLength |= d[0] << ((1-i) * 8); 4865 d = d[1 .. $]; 4866 } 4867 } else if(msg.lengthIndicator == 0x7f) { 4868 // 64 bit length 4869 msg.realLength = 0; 4870 4871 foreach(i; 0 .. 8) { 4872 msg.realLength |= d[0] << ((7-i) * 8); 4873 d = d[1 .. $]; 4874 } 4875 } else { 4876 // 7 bit length 4877 msg.realLength = msg.lengthIndicator; 4878 } 4879 4880 if(msg.masked) { 4881 msg.maskingKey = d[0 .. 4]; 4882 d = d[4 .. $]; 4883 } 4884 4885 msg.data = d[0 .. $]; 4886 4887 if(msg.masked) { 4888 // let's just unmask it now 4889 int keyIdx = 0; 4890 foreach(i; 0 .. msg.data.length) { 4891 msg.data[i] = msg.data[i] ^ msg.maskingKey[keyIdx]; 4892 if(keyIdx == 3) 4893 keyIdx = 0; 4894 else 4895 keyIdx++; 4896 } 4897 } 4898 4899 return msg; 4900 } 4901 4902 char[] textData() { 4903 return cast(char[]) data; 4904 } 4905 } 4906 4907 } 4908 4909 4910 version(Windows) 4911 { 4912 version(CRuntime_DigitalMars) 4913 { 4914 extern(C) int setmode(int, int) nothrow @nogc; 4915 } 4916 else version(CRuntime_Microsoft) 4917 { 4918 extern(C) int _setmode(int, int) nothrow @nogc; 4919 alias setmode = _setmode; 4920 } 4921 else static assert(0); 4922 } 4923 4924 version(Posix) { 4925 version(CRuntime_Musl) {} else { 4926 import core.sys.posix.unistd; 4927 private extern(C) int posix_spawn(pid_t*, const char*, void*, void*, const char**, const char**); 4928 } 4929 } 4930 4931 4932 // FIXME: these aren't quite public yet. 4933 //private: 4934 4935 // template for laziness 4936 void startWebsocketServer()() { 4937 version(linux) { 4938 import core.sys.posix.unistd; 4939 pid_t pid; 4940 const(char)*[16] args; 4941 args[0] = "ARSD_CGI_WEBSOCKET_SERVER"; 4942 args[1] = "--websocket-server"; 4943 posix_spawn(&pid, "/proc/self/exe", 4944 null, 4945 null, 4946 args.ptr, 4947 null // env 4948 ); 4949 } else version(Windows) { 4950 wchar[2048] filename; 4951 auto len = GetModuleFileNameW(null, filename.ptr, cast(DWORD) filename.length); 4952 if(len == 0 || len == filename.length) 4953 throw new Exception("could not get process name to start helper server"); 4954 4955 STARTUPINFOW startupInfo; 4956 startupInfo.cb = cast(DWORD) startupInfo.sizeof; 4957 PROCESS_INFORMATION processInfo; 4958 4959 // I *MIGHT* need to run it as a new job or a service... 4960 auto ret = CreateProcessW( 4961 filename.ptr, 4962 "--websocket-server"w, 4963 null, // process attributes 4964 null, // thread attributes 4965 false, // inherit handles 4966 0, // creation flags 4967 null, // environment 4968 null, // working directory 4969 &startupInfo, 4970 &processInfo 4971 ); 4972 4973 if(!ret) 4974 throw new Exception("create process failed"); 4975 4976 // when done with those, if we set them 4977 /* 4978 CloseHandle(hStdInput); 4979 CloseHandle(hStdOutput); 4980 CloseHandle(hStdError); 4981 */ 4982 4983 } else static assert(0, "Websocket server not implemented on this system yet (email me, i can prolly do it if you need it)"); 4984 } 4985 4986 // template for laziness 4987 /* 4988 The websocket server is a single-process, single-thread, event 4989 I/O thing. It is passed websockets from other CGI processes 4990 and is then responsible for handling their messages and responses. 4991 Note that the CGI process is responsible for websocket setup, 4992 including authentication, etc. 4993 4994 It also gets data sent to it by other processes and is responsible 4995 for distributing that, as necessary. 4996 */ 4997 void runWebsocketServer()() { 4998 assert(0, "not implemented"); 4999 } 5000 5001 void sendToWebsocketServer(WebSocket ws, string group) { 5002 assert(0, "not implemented"); 5003 } 5004 5005 void sendToWebsocketServer(string content, string group) { 5006 assert(0, "not implemented"); 5007 } 5008 5009 5010 void runEventServer()() { 5011 runAddonServer("/tmp/arsd_cgi_event_server", new EventSourceServerImplementation()); 5012 } 5013 5014 void runTimerServer()() { 5015 runAddonServer("/tmp/arsd_scheduled_job_server", new ScheduledJobServerImplementation()); 5016 } 5017 5018 version(Posix) { 5019 alias LocalServerConnectionHandle = int; 5020 alias CgiConnectionHandle = int; 5021 alias SocketConnectionHandle = int; 5022 5023 enum INVALID_CGI_CONNECTION_HANDLE = -1; 5024 } else version(Windows) { 5025 alias LocalServerConnectionHandle = HANDLE; 5026 version(embedded_httpd_threads) { 5027 alias CgiConnectionHandle = SOCKET; 5028 enum INVALID_CGI_CONNECTION_HANDLE = INVALID_SOCKET; 5029 } else version(fastcgi) { 5030 alias CgiConnectionHandle = void*; // Doesn't actually work! But I don't want compile to fail pointlessly at this point. 5031 enum INVALID_CGI_CONNECTION_HANDLE = null; 5032 } else version(scgi) { 5033 alias CgiConnectionHandle = SOCKET; 5034 enum INVALID_CGI_CONNECTION_HANDLE = INVALID_SOCKET; 5035 } else { /* version(plain_cgi) */ 5036 alias CgiConnectionHandle = HANDLE; 5037 enum INVALID_CGI_CONNECTION_HANDLE = null; 5038 } 5039 alias SocketConnectionHandle = SOCKET; 5040 } 5041 5042 version(with_addon_servers_connections) 5043 LocalServerConnectionHandle openLocalServerConnection(string name) { 5044 version(Posix) { 5045 import core.sys.posix.unistd; 5046 import core.sys.posix.sys.un; 5047 5048 int sock = socket(AF_UNIX, SOCK_STREAM, 0); 5049 if(sock == -1) 5050 throw new Exception("socket " ~ to!string(errno)); 5051 5052 scope(failure) 5053 close(sock); 5054 5055 // add-on server processes are assumed to be local, and thus will 5056 // use unix domain sockets. Besides, I want to pass sockets to them, 5057 // so it basically must be local (except for the session server, but meh). 5058 sockaddr_un addr; 5059 addr.sun_family = AF_UNIX; 5060 version(linux) { 5061 // on linux, we will use the abstract namespace 5062 addr.sun_path[0] = 0; 5063 addr.sun_path[1 .. name.length + 1] = cast(typeof(addr.sun_path[])) name[]; 5064 } else { 5065 // but otherwise, just use a file cuz we must. 5066 addr.sun_path[0 .. name.length] = cast(typeof(addr.sun_path[])) name[]; 5067 } 5068 5069 if(connect(sock, cast(sockaddr*) &addr, addr.sizeof) == -1) 5070 throw new Exception("connect " ~ to!string(errno)); 5071 5072 return sock; 5073 } else version(Windows) { 5074 return null; // FIXME 5075 } 5076 } 5077 5078 version(with_addon_servers_connections) 5079 void closeLocalServerConnection(LocalServerConnectionHandle handle) { 5080 version(Posix) { 5081 import core.sys.posix.unistd; 5082 close(handle); 5083 } else version(Windows) 5084 CloseHandle(handle); 5085 } 5086 5087 void runSessionServer()() { 5088 runAddonServer("/tmp/arsd_session_server", new BasicDataServerImplementation()); 5089 } 5090 5091 version(Posix) 5092 private void makeNonBlocking(int fd) { 5093 import core.sys.posix.fcntl; 5094 auto flags = fcntl(fd, F_GETFL, 0); 5095 if(flags == -1) 5096 throw new Exception("fcntl get"); 5097 flags |= O_NONBLOCK; 5098 auto s = fcntl(fd, F_SETFL, flags); 5099 if(s == -1) 5100 throw new Exception("fcntl set"); 5101 } 5102 5103 import core.stdc.errno; 5104 5105 struct IoOp { 5106 @disable this(); 5107 @disable this(this); 5108 5109 /* 5110 So we want to be able to eventually handle generic sockets too. 5111 */ 5112 5113 enum Read = 1; 5114 enum Write = 2; 5115 enum Accept = 3; 5116 enum ReadSocketHandle = 4; 5117 5118 // Your handler may be called in a different thread than the one that initiated the IO request! 5119 // It is also possible to have multiple io requests being called simultaneously. Use proper thread safety caution. 5120 private bool delegate(IoOp*, int) handler; // returns true if you are done and want it to be closed 5121 private void delegate(IoOp*) closeHandler; 5122 private void delegate(IoOp*) completeHandler; 5123 private int internalFd; 5124 private int operation; 5125 private int bufferLengthAllocated; 5126 private int bufferLengthUsed; 5127 private ubyte[1] internalBuffer; // it can be overallocated! 5128 5129 ubyte[] allocatedBuffer() { 5130 return internalBuffer.ptr[0 .. bufferLengthAllocated]; 5131 } 5132 5133 ubyte[] usedBuffer() { 5134 return allocatedBuffer[0 .. bufferLengthUsed]; 5135 } 5136 5137 void reset() { 5138 bufferLengthUsed = 0; 5139 } 5140 5141 int fd() { 5142 return internalFd; 5143 } 5144 } 5145 5146 IoOp* allocateIoOp(int fd, int operation, int bufferSize, bool delegate(IoOp*, int) handler) { 5147 import core.stdc.stdlib; 5148 5149 auto ptr = calloc(IoOp.sizeof + bufferSize, 1); 5150 if(ptr is null) 5151 assert(0); // out of memory! 5152 5153 auto op = cast(IoOp*) ptr; 5154 5155 op.handler = handler; 5156 op.internalFd = fd; 5157 op.operation = operation; 5158 op.bufferLengthAllocated = bufferSize; 5159 op.bufferLengthUsed = 0; 5160 5161 import core.memory; 5162 5163 GC.addRoot(ptr); 5164 5165 return op; 5166 } 5167 5168 void freeIoOp(ref IoOp* ptr) { 5169 5170 import core.memory; 5171 GC.removeRoot(ptr); 5172 5173 import core.stdc.stdlib; 5174 free(ptr); 5175 ptr = null; 5176 } 5177 5178 version(Posix) 5179 version(with_addon_servers_connections) 5180 void nonBlockingWrite(EventIoServer eis, int connection, const void[] data) { 5181 import core.sys.posix.unistd; 5182 5183 auto ret = write(connection, data.ptr, data.length); 5184 if(ret != data.length) { 5185 if(ret == 0 || errno == EPIPE) { 5186 // the file is closed, remove it 5187 eis.fileClosed(connection); 5188 } else 5189 throw new Exception("alas " ~ to!string(ret) ~ " " ~ to!string(errno)); // FIXME 5190 } 5191 } 5192 version(Windows) 5193 version(with_addon_servers_connections) 5194 void nonBlockingWrite(EventIoServer eis, int connection, const void[] data) { 5195 // FIXME 5196 } 5197 5198 bool isInvalidHandle(CgiConnectionHandle h) { 5199 return h == INVALID_CGI_CONNECTION_HANDLE; 5200 } 5201 5202 /+ 5203 https://docs.microsoft.com/en-us/windows/desktop/api/winsock2/nf-winsock2-wsarecv 5204 https://support.microsoft.com/en-gb/help/181611/socket-overlapped-i-o-versus-blocking-nonblocking-mode 5205 https://stackoverflow.com/questions/18018489/should-i-use-iocps-or-overlapped-wsasend-receive 5206 https://docs.microsoft.com/en-us/windows/desktop/fileio/i-o-completion-ports 5207 https://docs.microsoft.com/en-us/windows/desktop/fileio/createiocompletionport 5208 https://docs.microsoft.com/en-us/windows/desktop/api/mswsock/nf-mswsock-acceptex 5209 https://docs.microsoft.com/en-us/windows/desktop/Sync/waitable-timer-objects 5210 https://docs.microsoft.com/en-us/windows/desktop/api/synchapi/nf-synchapi-setwaitabletimer 5211 https://docs.microsoft.com/en-us/windows/desktop/Sync/using-a-waitable-timer-with-an-asynchronous-procedure-call 5212 https://docs.microsoft.com/en-us/windows/desktop/api/winsock2/nf-winsock2-wsagetoverlappedresult 5213 5214 +/ 5215 5216 /++ 5217 You can customize your server by subclassing the appropriate server. Then, register your 5218 subclass at compile time with the [registerEventIoServer] template, or implement your own 5219 main function and call it yourself. 5220 5221 $(TIP If you make your subclass a `final class`, there is a slight performance improvement.) 5222 +/ 5223 version(with_addon_servers_connections) 5224 interface EventIoServer { 5225 bool handleLocalConnectionData(IoOp* op, int receivedFd); 5226 void handleLocalConnectionClose(IoOp* op); 5227 void handleLocalConnectionComplete(IoOp* op); 5228 void wait_timeout(); 5229 void fileClosed(int fd); 5230 5231 void epoll_fd(int fd); 5232 } 5233 5234 // the sink should buffer it 5235 private void serialize(T)(scope void delegate(ubyte[]) sink, T t) { 5236 static if(is(T == struct)) { 5237 foreach(member; __traits(allMembers, T)) 5238 serialize(sink, __traits(getMember, t, member)); 5239 } else static if(is(T : int)) { 5240 // no need to think of endianness just because this is only used 5241 // for local, same-machine stuff anyway. thanks private lol 5242 sink((cast(ubyte*) &t)[0 .. t.sizeof]); 5243 } else static if(is(T == string) || is(T : const(ubyte)[])) { 5244 // these are common enough to optimize 5245 int len = cast(int) t.length; // want length consistent size tho, in case 32 bit program sends to 64 bit server, etc. 5246 sink((cast(ubyte*) &len)[0 .. int.sizeof]); 5247 sink(cast(ubyte[]) t[]); 5248 } else static if(is(T : A[], A)) { 5249 // generic array is less optimal but still prolly ok 5250 int len = cast(int) t.length; 5251 sink((cast(ubyte*) &len)[0 .. int.sizeof]); 5252 foreach(item; t) 5253 serialize(sink, item); 5254 } else static assert(0, T.stringof); 5255 } 5256 5257 // all may be stack buffers, so use cautio 5258 private void deserialize(T)(scope ubyte[] delegate(int sz) get, scope void delegate(T) dg) { 5259 static if(is(T == struct)) { 5260 T t; 5261 foreach(member; __traits(allMembers, T)) 5262 deserialize!(typeof(__traits(getMember, T, member)))(get, (mbr) { __traits(getMember, t, member) = mbr; }); 5263 dg(t); 5264 } else static if(is(T : int)) { 5265 // no need to think of endianness just because this is only used 5266 // for local, same-machine stuff anyway. thanks private lol 5267 T t; 5268 auto data = get(t.sizeof); 5269 t = (cast(T[]) data)[0]; 5270 dg(t); 5271 } else static if(is(T == string) || is(T : const(ubyte)[])) { 5272 // these are common enough to optimize 5273 int len; 5274 auto data = get(len.sizeof); 5275 len = (cast(int[]) data)[0]; 5276 5277 /* 5278 typeof(T[0])[2000] stackBuffer; 5279 T buffer; 5280 5281 if(len < stackBuffer.length) 5282 buffer = stackBuffer[0 .. len]; 5283 else 5284 buffer = new T(len); 5285 5286 data = get(len * typeof(T[0]).sizeof); 5287 */ 5288 5289 T t = cast(T) get(len * cast(int) typeof(T.init[0]).sizeof); 5290 5291 dg(t); 5292 } else static if(is(T == E[], E)) { 5293 T t; 5294 int len; 5295 auto data = get(len.sizeof); 5296 len = (cast(int[]) data)[0]; 5297 t.length = len; 5298 foreach(ref e; t) { 5299 deserialize!E(get, (ele) { e = ele; }); 5300 } 5301 dg(t); 5302 } else static assert(0, T.stringof); 5303 } 5304 5305 unittest { 5306 serialize((ubyte[] b) { 5307 deserialize!int( sz => b[0 .. sz], (t) { assert(t == 1); }); 5308 }, 1); 5309 serialize((ubyte[] b) { 5310 deserialize!int( sz => b[0 .. sz], (t) { assert(t == 56674); }); 5311 }, 56674); 5312 ubyte[1000] buffer; 5313 int bufferPoint; 5314 void add(ubyte[] b) { 5315 buffer[bufferPoint .. bufferPoint + b.length] = b[]; 5316 bufferPoint += b.length; 5317 } 5318 ubyte[] get(int sz) { 5319 auto b = buffer[bufferPoint .. bufferPoint + sz]; 5320 bufferPoint += sz; 5321 return b; 5322 } 5323 serialize(&add, "test here"); 5324 bufferPoint = 0; 5325 deserialize!string(&get, (t) { assert(t == "test here"); }); 5326 bufferPoint = 0; 5327 5328 struct Foo { 5329 int a; 5330 ubyte c; 5331 string d; 5332 } 5333 serialize(&add, Foo(403, 37, "amazing")); 5334 bufferPoint = 0; 5335 deserialize!Foo(&get, (t) { 5336 assert(t.a == 403); 5337 assert(t.c == 37); 5338 assert(t.d == "amazing"); 5339 }); 5340 bufferPoint = 0; 5341 } 5342 5343 /* 5344 Here's the way the RPC interface works: 5345 5346 You define the interface that lists the functions you can call on the remote process. 5347 The interface may also have static methods for convenience. These forward to a singleton 5348 instance of an auto-generated class, which actually sends the args over the pipe. 5349 5350 An impl class actually implements it. A receiving server deserializes down the pipe and 5351 calls methods on the class. 5352 5353 I went with the interface to get some nice compiler checking and documentation stuff. 5354 5355 I could have skipped the interface and just implemented it all from the server class definition 5356 itself, but then the usage may call the method instead of rpcing it; I just like having the user 5357 interface and the implementation separate so you aren't tempted to `new impl` to call the methods. 5358 5359 5360 I fiddled with newlines in the mixin string to ensure the assert line numbers matched up to the source code line number. Idk why dmd didn't do this automatically, but it was important to me. 5361 5362 Realistically though the bodies would just be 5363 connection.call(this.mangleof, args...) sooooo. 5364 5365 FIXME: overloads aren't supported 5366 */ 5367 5368 /// Base for storing sessions in an array. Exists primarily for internal purposes and you should generally not use this. 5369 interface SessionObject {} 5370 5371 private immutable void delegate(string[])[string] scheduledJobHandlers; 5372 5373 version(with_breaking_cgi_features) 5374 mixin(q{ 5375 5376 mixin template ImplementRpcClientInterface(T, string serverPath) { 5377 static import std.traits; 5378 5379 // derivedMembers on an interface seems to give exactly what I want: the virtual functions we need to implement. so I am just going to use it directly without more filtering. 5380 static foreach(idx, member; __traits(derivedMembers, T)) { 5381 static if(__traits(isVirtualFunction, __traits(getMember, T, member))) 5382 mixin( q{ 5383 std.traits.ReturnType!(__traits(getMember, T, member)) 5384 } ~ member ~ q{(std.traits.Parameters!(__traits(getMember, T, member)) params) 5385 { 5386 SerializationBuffer buffer; 5387 auto i = cast(ushort) idx; 5388 serialize(&buffer.sink, i); 5389 serialize(&buffer.sink, __traits(getMember, T, member).mangleof); 5390 foreach(param; params) 5391 serialize(&buffer.sink, param); 5392 5393 auto sendable = buffer.sendable; 5394 5395 version(Posix) {{ 5396 auto ret = send(connectionHandle, sendable.ptr, sendable.length, 0); 5397 assert(ret == sendable.length); 5398 }} // FIXME Windows impl 5399 5400 static if(!is(typeof(return) == void)) { 5401 // there is a return value; we need to wait for it too 5402 version(Posix) { 5403 ubyte[3000] revBuffer; 5404 auto ret = recv(connectionHandle, revBuffer.ptr, revBuffer.length, 0); 5405 auto got = revBuffer[0 .. ret]; 5406 5407 int dataLocation; 5408 ubyte[] grab(int sz) { 5409 auto d = got[dataLocation .. dataLocation + sz]; 5410 dataLocation += sz; 5411 return d; 5412 } 5413 5414 typeof(return) retu; 5415 deserialize!(typeof(return))(&grab, (a) { retu = a; }); 5416 return retu; 5417 } else { 5418 // FIXME Windows impl 5419 return typeof(return).init; 5420 } 5421 5422 } 5423 }}); 5424 } 5425 5426 private static typeof(this) singletonInstance; 5427 private LocalServerConnectionHandle connectionHandle; 5428 5429 static typeof(this) connection() { 5430 if(singletonInstance is null) { 5431 singletonInstance = new typeof(this)(); 5432 singletonInstance.connect(); 5433 } 5434 return singletonInstance; 5435 } 5436 5437 void connect() { 5438 connectionHandle = openLocalServerConnection(serverPath); 5439 } 5440 5441 void disconnect() { 5442 closeLocalServerConnection(connectionHandle); 5443 } 5444 } 5445 5446 void dispatchRpcServer(Interface, Class)(Class this_, ubyte[] data, int fd) if(is(Class : Interface)) { 5447 ushort calledIdx; 5448 string calledFunction; 5449 5450 int dataLocation; 5451 ubyte[] grab(int sz) { 5452 auto d = data[dataLocation .. dataLocation + sz]; 5453 dataLocation += sz; 5454 return d; 5455 } 5456 5457 again: 5458 5459 deserialize!ushort(&grab, (a) { calledIdx = a; }); 5460 deserialize!string(&grab, (a) { calledFunction = a; }); 5461 5462 import std.traits; 5463 5464 sw: switch(calledIdx) { 5465 static foreach(idx, memberName; __traits(derivedMembers, Interface)) 5466 static if(__traits(isVirtualFunction, __traits(getMember, Interface, memberName))) { 5467 case idx: 5468 assert(calledFunction == __traits(getMember, Interface, memberName).mangleof); 5469 5470 Parameters!(__traits(getMember, Interface, memberName)) params; 5471 foreach(ref param; params) 5472 deserialize!(typeof(param))(&grab, (a) { param = a; }); 5473 5474 static if(is(ReturnType!(__traits(getMember, Interface, memberName)) == void)) { 5475 __traits(getMember, this_, memberName)(params); 5476 } else { 5477 auto ret = __traits(getMember, this_, memberName)(params); 5478 SerializationBuffer buffer; 5479 serialize(&buffer.sink, ret); 5480 5481 auto sendable = buffer.sendable; 5482 5483 version(Posix) { 5484 auto r = send(fd, sendable.ptr, sendable.length, 0); 5485 assert(r == sendable.length); 5486 } // FIXME Windows impl 5487 } 5488 break sw; 5489 } 5490 default: assert(0); 5491 } 5492 5493 if(dataLocation != data.length) 5494 goto again; 5495 } 5496 5497 5498 private struct SerializationBuffer { 5499 ubyte[2048] bufferBacking; 5500 int bufferLocation; 5501 void sink(scope ubyte[] data) { 5502 bufferBacking[bufferLocation .. bufferLocation + data.length] = data[]; 5503 bufferLocation += data.length; 5504 } 5505 5506 ubyte[] sendable() { 5507 return bufferBacking[0 .. bufferLocation]; 5508 } 5509 } 5510 5511 /* 5512 FIXME: 5513 add a version command line arg 5514 version data in the library 5515 management gui as external program 5516 5517 at server with event_fd for each run 5518 use .mangleof in the at function name 5519 5520 i think the at server will have to: 5521 pipe args to the child 5522 collect child output for logging 5523 get child return value for logging 5524 5525 on windows timers work differently. idk how to best combine with the io stuff. 5526 5527 will have to have dump and restore too, so i can restart without losing stuff. 5528 */ 5529 5530 /++ 5531 A convenience object for talking to the [BasicDataServer] from a higher level. 5532 See: [Cgi.getSessionObject]. 5533 5534 You pass it a `Data` struct describing the data you want saved in the session. 5535 Then, this class will generate getter and setter properties that allow access 5536 to that data. 5537 5538 Note that each load and store will be done as-accessed; it doesn't front-load 5539 mutable data nor does it batch updates out of fear of read-modify-write race 5540 conditions. (In fact, right now it does this for everything, but in the future, 5541 I might batch load `immutable` members of the Data struct.) 5542 5543 At some point in the future, I might also let it do different backends, like 5544 a client-side cookie store too, but idk. 5545 5546 Note that the plain-old-data members of your `Data` struct are wrapped by this 5547 interface via a static foreach to make property functions. 5548 5549 See_Also: [MockSession] 5550 +/ 5551 interface Session(Data) : SessionObject { 5552 @property string sessionId() const; 5553 5554 /++ 5555 Starts a new session. Note that a session is also 5556 implicitly started as soon as you write data to it, 5557 so if you need to alter these parameters from their 5558 defaults, be sure to explicitly call this BEFORE doing 5559 any writes to session data. 5560 5561 Params: 5562 idleLifetime = How long, in seconds, the session 5563 should remain in memory when not being read from 5564 or written to. The default is one day. 5565 5566 NOT IMPLEMENTED 5567 5568 useExtendedLifetimeCookie = The session ID is always 5569 stored in a HTTP cookie, and by default, that cookie 5570 is discarded when the user closes their browser. 5571 5572 But if you set this to true, it will use a non-perishable 5573 cookie for the given idleLifetime. 5574 5575 NOT IMPLEMENTED 5576 +/ 5577 void start(int idleLifetime = 2600 * 24, bool useExtendedLifetimeCookie = false); 5578 5579 /++ 5580 Regenerates the session ID and updates the associated 5581 cookie. 5582 5583 This is also your chance to change immutable data 5584 (not yet implemented). 5585 +/ 5586 void regenerateId(); 5587 5588 /++ 5589 Terminates this session, deleting all saved data. 5590 +/ 5591 void terminate(); 5592 5593 /++ 5594 Plain-old-data members of your `Data` struct are wrapped here via 5595 the property getters and setters. 5596 5597 If the member is a non-string array, it returns a magical array proxy 5598 object which allows for atomic appends and replaces via overloaded operators. 5599 You can slice this to get a range representing a $(B const) view of the array. 5600 This is to protect you against read-modify-write race conditions. 5601 +/ 5602 static foreach(memberName; __traits(allMembers, Data)) 5603 static if(is(typeof(__traits(getMember, Data, memberName)))) 5604 mixin(q{ 5605 @property inout(typeof(__traits(getMember, Data, memberName))) } ~ memberName ~ q{ () inout; 5606 @property void } ~ memberName ~ q{ (typeof(__traits(getMember, Data, memberName)) value); 5607 }); 5608 5609 } 5610 5611 /++ 5612 An implementation of [Session] that works on real cgi connections utilizing the 5613 [BasicDataServer]. 5614 5615 As opposed to a [MockSession] which is made for testing purposes. 5616 5617 You will not construct one of these directly. See [Cgi.getSessionObject] instead. 5618 +/ 5619 class BasicDataServerSession(Data) : Session!Data { 5620 private Cgi cgi; 5621 private string sessionId_; 5622 5623 public @property string sessionId() const { 5624 return sessionId_; 5625 } 5626 5627 protected @property string sessionId(string s) { 5628 return this.sessionId_ = s; 5629 } 5630 5631 private this(Cgi cgi) { 5632 this.cgi = cgi; 5633 if(auto ptr = "sessionId" in cgi.cookies) 5634 sessionId = (*ptr).length ? *ptr : null; 5635 } 5636 5637 void start(int idleLifetime = 2600 * 24, bool useExtendedLifetimeCookie = false) { 5638 assert(sessionId is null); 5639 5640 // FIXME: what if there is a session ID cookie, but no corresponding session on the server? 5641 5642 import std.random, std.conv; 5643 sessionId = to!string(uniform(1, long.max)); 5644 5645 BasicDataServer.connection.createSession(sessionId, idleLifetime); 5646 setCookie(); 5647 } 5648 5649 protected void setCookie() { 5650 cgi.setCookie( 5651 "sessionId", sessionId, 5652 0 /* expiration */, 5653 "/" /* path */, 5654 null /* domain */, 5655 true /* http only */, 5656 cgi.https /* if the session is started on https, keep it there, otherwise, be flexible */); 5657 } 5658 5659 void regenerateId() { 5660 if(sessionId is null) { 5661 start(); 5662 return; 5663 } 5664 import std.random, std.conv; 5665 auto oldSessionId = sessionId; 5666 sessionId = to!string(uniform(1, long.max)); 5667 BasicDataServer.connection.renameSession(oldSessionId, sessionId); 5668 setCookie(); 5669 } 5670 5671 void terminate() { 5672 BasicDataServer.connection.destroySession(sessionId); 5673 sessionId = null; 5674 setCookie(); 5675 } 5676 5677 static foreach(memberName; __traits(allMembers, Data)) 5678 static if(is(typeof(__traits(getMember, Data, memberName)))) 5679 mixin(q{ 5680 @property inout(typeof(__traits(getMember, Data, memberName))) } ~ memberName ~ q{ () inout { 5681 if(sessionId is null) 5682 return typeof(return).init; 5683 5684 import std.traits; 5685 auto v = BasicDataServer.connection.getSessionData(sessionId, fullyQualifiedName!Data ~ "." ~ memberName); 5686 if(v.length == 0) 5687 return typeof(return).init; 5688 import std.conv; 5689 // why this cast? to doesn't like being given an inout argument. so need to do it without that, then 5690 // we need to return it and that needed the cast. It should be fine since we basically respect constness.. 5691 // basically. Assuming the session is POD this should be fine. 5692 return cast(typeof(return)) to!(typeof(__traits(getMember, Data, memberName)))(v); 5693 } 5694 @property void } ~ memberName ~ q{ (typeof(__traits(getMember, Data, memberName)) value) { 5695 if(sessionId is null) 5696 start(); 5697 import std.conv; 5698 import std.traits; 5699 BasicDataServer.connection.setSessionData(sessionId, fullyQualifiedName!Data ~ "." ~ memberName, to!string(value)); 5700 } 5701 }); 5702 } 5703 5704 /++ 5705 A mock object that works like the real session, but doesn't actually interact with any actual database or http connection. 5706 Simply stores the data in its instance members. 5707 +/ 5708 class MockSession(Data) : Session!Data { 5709 pure { 5710 @property string sessionId() const { return "mock"; } 5711 void start(int idleLifetime = 2600 * 24, bool useExtendedLifetimeCookie = false) {} 5712 void regenerateId() {} 5713 void terminate() {} 5714 5715 private Data store_; 5716 5717 static foreach(memberName; __traits(allMembers, Data)) 5718 static if(is(typeof(__traits(getMember, Data, memberName)))) 5719 mixin(q{ 5720 @property inout(typeof(__traits(getMember, Data, memberName))) } ~ memberName ~ q{ () inout { 5721 return __traits(getMember, store_, memberName); 5722 } 5723 @property void } ~ memberName ~ q{ (typeof(__traits(getMember, Data, memberName)) value) { 5724 __traits(getMember, store_, memberName) = value; 5725 } 5726 }); 5727 } 5728 } 5729 5730 /++ 5731 Direct interface to the basic data add-on server. You can 5732 typically use [Cgi.getSessionObject] as a more convenient interface. 5733 +/ 5734 version(with_addon_servers_connections) 5735 interface BasicDataServer { 5736 /// 5737 void createSession(string sessionId, int lifetime); 5738 /// 5739 void renewSession(string sessionId, int lifetime); 5740 /// 5741 void destroySession(string sessionId); 5742 /// 5743 void renameSession(string oldSessionId, string newSessionId); 5744 5745 /// 5746 void setSessionData(string sessionId, string dataKey, string dataValue); 5747 /// 5748 string getSessionData(string sessionId, string dataKey); 5749 5750 /// 5751 static BasicDataServerConnection connection() { 5752 return BasicDataServerConnection.connection(); 5753 } 5754 } 5755 5756 version(with_addon_servers_connections) 5757 class BasicDataServerConnection : BasicDataServer { 5758 mixin ImplementRpcClientInterface!(BasicDataServer, "/tmp/arsd_session_server"); 5759 } 5760 5761 version(with_addon_servers) 5762 final class BasicDataServerImplementation : BasicDataServer, EventIoServer { 5763 5764 void createSession(string sessionId, int lifetime) { 5765 sessions[sessionId.idup] = Session(lifetime); 5766 } 5767 void destroySession(string sessionId) { 5768 sessions.remove(sessionId); 5769 } 5770 void renewSession(string sessionId, int lifetime) { 5771 sessions[sessionId].lifetime = lifetime; 5772 } 5773 void renameSession(string oldSessionId, string newSessionId) { 5774 sessions[newSessionId.idup] = sessions[oldSessionId]; 5775 sessions.remove(oldSessionId); 5776 } 5777 void setSessionData(string sessionId, string dataKey, string dataValue) { 5778 if(sessionId !in sessions) 5779 createSession(sessionId, 3600); // FIXME? 5780 sessions[sessionId].values[dataKey.idup] = dataValue.idup; 5781 } 5782 string getSessionData(string sessionId, string dataKey) { 5783 if(auto session = sessionId in sessions) { 5784 if(auto data = dataKey in (*session).values) 5785 return *data; 5786 else 5787 return null; // no such data 5788 5789 } else { 5790 return null; // no session 5791 } 5792 } 5793 5794 5795 protected: 5796 5797 struct Session { 5798 int lifetime; 5799 5800 string[string] values; 5801 } 5802 5803 Session[string] sessions; 5804 5805 bool handleLocalConnectionData(IoOp* op, int receivedFd) { 5806 auto data = op.usedBuffer; 5807 dispatchRpcServer!BasicDataServer(this, data, op.fd); 5808 return false; 5809 } 5810 5811 void handleLocalConnectionClose(IoOp* op) {} // doesn't really matter, this is a fairly stateless go 5812 void handleLocalConnectionComplete(IoOp* op) {} // again, irrelevant 5813 void wait_timeout() {} 5814 void fileClosed(int fd) {} // stateless so irrelevant 5815 void epoll_fd(int fd) {} 5816 } 5817 5818 /++ 5819 See [schedule] to make one of these. You then call one of the methods here to set it up: 5820 5821 --- 5822 schedule!fn(args).at(DateTime(2019, 8, 7, 12, 00, 00)); // run the function at August 7, 2019, 12 noon UTC 5823 schedule!fn(args).delay(6.seconds); // run it after waiting 6 seconds 5824 schedule!fn(args).asap(); // run it in the background as soon as the event loop gets around to it 5825 --- 5826 +/ 5827 version(with_addon_servers_connections) 5828 struct ScheduledJobHelper { 5829 private string func; 5830 private string[] args; 5831 private bool consumed; 5832 5833 private this(string func, string[] args) { 5834 this.func = func; 5835 this.args = args; 5836 } 5837 5838 ~this() { 5839 assert(consumed); 5840 } 5841 5842 /++ 5843 Schedules the job to be run at the given time. 5844 +/ 5845 void at(DateTime when, immutable TimeZone timezone = UTC()) { 5846 consumed = true; 5847 5848 auto conn = ScheduledJobServerConnection.connection; 5849 import std.file; 5850 auto st = SysTime(when, timezone); 5851 auto jobId = conn.scheduleJob(1, cast(int) st.toUnixTime(), thisExePath, func, args); 5852 } 5853 5854 /++ 5855 Schedules the job to run at least after the specified delay. 5856 +/ 5857 void delay(Duration delay) { 5858 consumed = true; 5859 5860 auto conn = ScheduledJobServerConnection.connection; 5861 import std.file; 5862 auto jobId = conn.scheduleJob(0, cast(int) delay.total!"seconds", thisExePath, func, args); 5863 } 5864 5865 /++ 5866 Runs the job in the background ASAP. 5867 5868 $(NOTE It may run in a background thread. Don't segfault!) 5869 +/ 5870 void asap() { 5871 consumed = true; 5872 5873 auto conn = ScheduledJobServerConnection.connection; 5874 import std.file; 5875 auto jobId = conn.scheduleJob(0, 1, thisExePath, func, args); 5876 } 5877 5878 /+ 5879 /++ 5880 Schedules the job to recur on the given pattern. 5881 +/ 5882 void recur(string spec) { 5883 5884 } 5885 +/ 5886 } 5887 5888 /++ 5889 First step to schedule a job on the scheduled job server. 5890 5891 The scheduled job needs to be a top-level function that doesn't read any 5892 variables from outside its arguments because it may be run in a new process, 5893 without any context existing later. 5894 5895 You MUST set details on the returned object to actually do anything! 5896 +/ 5897 template schedule(alias fn, T...) if(is(typeof(fn) == function)) { 5898 /// 5899 ScheduledJobHelper schedule(T args) { 5900 // this isn't meant to ever be called, but instead just to 5901 // get the compiler to type check the arguments passed for us 5902 auto sample = delegate() { 5903 fn(args); 5904 }; 5905 string[] sargs; 5906 foreach(arg; args) 5907 sargs ~= to!string(arg); 5908 return ScheduledJobHelper(fn.mangleof, sargs); 5909 } 5910 5911 shared static this() { 5912 scheduledJobHandlers[fn.mangleof] = delegate(string[] sargs) { 5913 import std.traits; 5914 Parameters!fn args; 5915 foreach(idx, ref arg; args) 5916 arg = to!(typeof(arg))(sargs[idx]); 5917 fn(args); 5918 }; 5919 } 5920 } 5921 5922 /// 5923 interface ScheduledJobServer { 5924 /// Use the [schedule] function for a higher-level interface. 5925 int scheduleJob(int whenIs, int when, string executable, string func, string[] args); 5926 /// 5927 void cancelJob(int jobId); 5928 } 5929 5930 version(with_addon_servers_connections) 5931 class ScheduledJobServerConnection : ScheduledJobServer { 5932 mixin ImplementRpcClientInterface!(ScheduledJobServer, "/tmp/arsd_scheduled_job_server"); 5933 } 5934 5935 version(with_addon_servers) 5936 final class ScheduledJobServerImplementation : ScheduledJobServer, EventIoServer { 5937 // FIXME: we need to handle SIGCHLD in this somehow 5938 // whenIs is 0 for relative, 1 for absolute 5939 protected int scheduleJob(int whenIs, int when, string executable, string func, string[] args) { 5940 auto nj = nextJobId; 5941 nextJobId++; 5942 5943 version(linux) { 5944 import core.sys.linux.timerfd; 5945 import core.sys.linux.epoll; 5946 import core.sys.posix.unistd; 5947 5948 5949 auto fd = timerfd_create(CLOCK_REALTIME, TFD_NONBLOCK | TFD_CLOEXEC); 5950 if(fd == -1) 5951 throw new Exception("fd timer create failed"); 5952 5953 auto job = Job(executable, func, args, fd, nj); 5954 5955 itimerspec value; 5956 value.it_value.tv_sec = when; 5957 value.it_value.tv_nsec = 0; 5958 5959 value.it_interval.tv_sec = 0; 5960 value.it_interval.tv_nsec = 0; 5961 5962 if(timerfd_settime(fd, whenIs == 1 ? TFD_TIMER_ABSTIME : 0, &value, null) == -1) 5963 throw new Exception("couldn't set fd timer"); 5964 5965 auto op = allocateIoOp(fd, IoOp.Read, 16, (IoOp* op, int fd) { 5966 jobs.remove(nj); 5967 5968 spawnProcess([job.executable, "--timed-job", job.func] ~ args); 5969 5970 return true; 5971 }); 5972 scope(failure) 5973 freeIoOp(op); 5974 5975 epoll_event ev; 5976 ev.events = EPOLLIN | EPOLLET; 5977 ev.data.ptr = op; 5978 if(epoll_ctl(epoll_fd, EPOLL_CTL_ADD, fd, &ev) == -1) 5979 throw new Exception("epoll_ctl " ~ to!string(errno)); 5980 5981 jobs[nj] = job; 5982 return nj; 5983 } else assert(0); 5984 } 5985 5986 protected void cancelJob(int jobId) { 5987 version(linux) { 5988 auto job = jobId in jobs; 5989 if(job is null) 5990 return; 5991 5992 version(linux) { 5993 import core.sys.linux.timerfd; 5994 import core.sys.linux.epoll; 5995 import core.sys.posix.unistd; 5996 epoll_ctl(epoll_fd, EPOLL_CTL_DEL, job.timerfd, null); 5997 close(job.timerfd); 5998 } 5999 } 6000 jobs.remove(jobId); 6001 } 6002 6003 int nextJobId = 1; 6004 static struct Job { 6005 string executable; 6006 string func; 6007 string[] args; 6008 int timerfd; 6009 int id; 6010 } 6011 Job[int] jobs; 6012 6013 6014 // event io server methods below 6015 6016 bool handleLocalConnectionData(IoOp* op, int receivedFd) { 6017 auto data = op.usedBuffer; 6018 dispatchRpcServer!ScheduledJobServer(this, data, op.fd); 6019 return false; 6020 } 6021 6022 void handleLocalConnectionClose(IoOp* op) {} // doesn't really matter, this is a fairly stateless go 6023 void handleLocalConnectionComplete(IoOp* op) {} // again, irrelevant 6024 void wait_timeout() {} 6025 void fileClosed(int fd) {} // stateless so irrelevant 6026 6027 int epoll_fd_; 6028 void epoll_fd(int fd) {this.epoll_fd_ = fd; } 6029 int epoll_fd() { return epoll_fd_; } 6030 } 6031 6032 /// 6033 version(with_addon_servers_connections) 6034 interface EventSourceServer { 6035 /++ 6036 sends this cgi request to the event server so it will be fed events. You should not do anything else with the cgi object after this. 6037 6038 $(WARNING This API is extremely unstable. I might change it or remove it without notice.) 6039 6040 See_Also: 6041 [sendEvent] 6042 +/ 6043 public static void adoptConnection(Cgi cgi, in char[] eventUrl) { 6044 /* 6045 If lastEventId is missing or empty, you just get new events as they come. 6046 6047 If it is set from something else, it sends all since then (that are still alive) 6048 down the pipe immediately. 6049 6050 The reason it can come from the header is that's what the standard defines for 6051 browser reconnects. The reason it can come from a query string is just convenience 6052 in catching up in a user-defined manner. 6053 6054 The reason the header overrides the query string is if the browser tries to reconnect, 6055 it will send the header AND the query (it reconnects to the same url), so we just 6056 want to do the restart thing. 6057 6058 Note that if you ask for "0" as the lastEventId, it will get ALL still living events. 6059 */ 6060 string lastEventId = cgi.lastEventId; 6061 if(lastEventId.length == 0 && "lastEventId" in cgi.get) 6062 lastEventId = cgi.get["lastEventId"]; 6063 6064 cgi.setResponseContentType("text/event-stream"); 6065 cgi.write(":\n", false); // to initialize the chunking and send headers before keeping the fd for later 6066 cgi.flush(); 6067 6068 cgi.closed = true; 6069 auto s = openLocalServerConnection("/tmp/arsd_cgi_event_server"); 6070 scope(exit) 6071 closeLocalServerConnection(s); 6072 6073 version(fastcgi) 6074 throw new Exception("sending fcgi connections not supported"); 6075 else { 6076 auto fd = cgi.getOutputFileHandle(); 6077 if(isInvalidHandle(fd)) 6078 throw new Exception("bad fd from cgi!"); 6079 6080 EventSourceServerImplementation.SendableEventConnection sec; 6081 sec.populate(cgi.responseChunked, eventUrl, lastEventId); 6082 6083 version(Posix) { 6084 auto res = write_fd(s, cast(void*) &sec, sec.sizeof, fd); 6085 assert(res == sec.sizeof); 6086 } else version(Windows) { 6087 // FIXME 6088 } 6089 } 6090 } 6091 6092 /++ 6093 Sends an event to the event server, starting it if necessary. The event server will distribute it to any listening clients, and store it for `lifetime` seconds for any later listening clients to catch up later. 6094 6095 $(WARNING This API is extremely unstable. I might change it or remove it without notice.) 6096 6097 Params: 6098 url = A string identifying this event "bucket". Listening clients must also connect to this same string. I called it `url` because I envision it being just passed as the url of the request. 6099 event = the event type string, which is used in the Javascript addEventListener API on EventSource 6100 data = the event data. Available in JS as `event.data`. 6101 lifetime = the amount of time to keep this event for replaying on the event server. 6102 6103 See_Also: 6104 [sendEventToEventServer] 6105 +/ 6106 public static void sendEvent(string url, string event, string data, int lifetime) { 6107 auto s = openLocalServerConnection("/tmp/arsd_cgi_event_server"); 6108 scope(exit) 6109 closeLocalServerConnection(s); 6110 6111 EventSourceServerImplementation.SendableEvent sev; 6112 sev.populate(url, event, data, lifetime); 6113 6114 version(Posix) { 6115 auto ret = send(s, &sev, sev.sizeof, 0); 6116 assert(ret == sev.sizeof); 6117 } else version(Windows) { 6118 // FIXME 6119 } 6120 } 6121 6122 /++ 6123 Messages sent to `url` will also be sent to anyone listening on `forwardUrl`. 6124 6125 See_Also: [disconnect] 6126 +/ 6127 void connect(string url, string forwardUrl); 6128 6129 /++ 6130 Disconnects `forwardUrl` from `url` 6131 6132 See_Also: [connect] 6133 +/ 6134 void disconnect(string url, string forwardUrl); 6135 } 6136 6137 /// 6138 version(with_addon_servers) 6139 final class EventSourceServerImplementation : EventSourceServer, EventIoServer { 6140 6141 protected: 6142 6143 void connect(string url, string forwardUrl) { 6144 pipes[url] ~= forwardUrl; 6145 } 6146 void disconnect(string url, string forwardUrl) { 6147 auto t = url in pipes; 6148 if(t is null) 6149 return; 6150 foreach(idx, n; (*t)) 6151 if(n == forwardUrl) { 6152 (*t)[idx] = (*t)[$-1]; 6153 (*t) = (*t)[0 .. $-1]; 6154 break; 6155 } 6156 } 6157 6158 bool handleLocalConnectionData(IoOp* op, int receivedFd) { 6159 if(receivedFd != -1) { 6160 //writeln("GOT FD ", receivedFd, " -- ", op.usedBuffer); 6161 6162 //core.sys.posix.unistd.write(receivedFd, "hello".ptr, 5); 6163 6164 SendableEventConnection* got = cast(SendableEventConnection*) op.usedBuffer.ptr; 6165 6166 auto url = got.url.idup; 6167 eventConnectionsByUrl[url] ~= EventConnection(receivedFd, got.responseChunked > 0 ? true : false); 6168 6169 // FIXME: catch up on past messages here 6170 } else { 6171 auto data = op.usedBuffer; 6172 auto event = cast(SendableEvent*) data.ptr; 6173 6174 if(event.magic == 0xdeadbeef) { 6175 handleInputEvent(event); 6176 6177 if(event.url in pipes) 6178 foreach(pipe; pipes[event.url]) { 6179 event.url = pipe; 6180 handleInputEvent(event); 6181 } 6182 } else { 6183 dispatchRpcServer!EventSourceServer(this, data, op.fd); 6184 } 6185 } 6186 return false; 6187 } 6188 void handleLocalConnectionClose(IoOp* op) {} 6189 void handleLocalConnectionComplete(IoOp* op) {} 6190 6191 void wait_timeout() { 6192 // just keeping alive 6193 foreach(url, connections; eventConnectionsByUrl) 6194 foreach(connection; connections) 6195 if(connection.needsChunking) 6196 nonBlockingWrite(this, connection.fd, "2\r\n:\n"); 6197 else 6198 nonBlockingWrite(this, connection.fd, ":\n"); 6199 } 6200 6201 void fileClosed(int fd) { 6202 outer: foreach(url, ref connections; eventConnectionsByUrl) { 6203 foreach(idx, conn; connections) { 6204 if(fd == conn.fd) { 6205 connections[idx] = connections[$-1]; 6206 connections = connections[0 .. $ - 1]; 6207 continue outer; 6208 } 6209 } 6210 } 6211 } 6212 6213 void epoll_fd(int fd) {} 6214 6215 6216 private: 6217 6218 6219 struct SendableEventConnection { 6220 ubyte responseChunked; 6221 6222 int urlLength; 6223 char[256] urlBuffer = 0; 6224 6225 int lastEventIdLength; 6226 char[32] lastEventIdBuffer = 0; 6227 6228 char[] url() { 6229 return urlBuffer[0 .. urlLength]; 6230 } 6231 void url(in char[] u) { 6232 urlBuffer[0 .. u.length] = u[]; 6233 urlLength = cast(int) u.length; 6234 } 6235 char[] lastEventId() { 6236 return lastEventIdBuffer[0 .. lastEventIdLength]; 6237 } 6238 void populate(bool responseChunked, in char[] url, in char[] lastEventId) 6239 in { 6240 assert(url.length < this.urlBuffer.length); 6241 assert(lastEventId.length < this.lastEventIdBuffer.length); 6242 } 6243 do { 6244 this.responseChunked = responseChunked ? 1 : 0; 6245 this.urlLength = cast(int) url.length; 6246 this.lastEventIdLength = cast(int) lastEventId.length; 6247 6248 this.urlBuffer[0 .. url.length] = url[]; 6249 this.lastEventIdBuffer[0 .. lastEventId.length] = lastEventId[]; 6250 } 6251 } 6252 6253 struct SendableEvent { 6254 int magic = 0xdeadbeef; 6255 int urlLength; 6256 char[256] urlBuffer = 0; 6257 int typeLength; 6258 char[32] typeBuffer = 0; 6259 int messageLength; 6260 char[2048] messageBuffer = 0; 6261 int _lifetime; 6262 6263 char[] message() { 6264 return messageBuffer[0 .. messageLength]; 6265 } 6266 char[] type() { 6267 return typeBuffer[0 .. typeLength]; 6268 } 6269 char[] url() { 6270 return urlBuffer[0 .. urlLength]; 6271 } 6272 void url(in char[] u) { 6273 urlBuffer[0 .. u.length] = u[]; 6274 urlLength = cast(int) u.length; 6275 } 6276 int lifetime() { 6277 return _lifetime; 6278 } 6279 6280 /// 6281 void populate(string url, string type, string message, int lifetime) 6282 in { 6283 assert(url.length < this.urlBuffer.length); 6284 assert(type.length < this.typeBuffer.length); 6285 assert(message.length < this.messageBuffer.length); 6286 } 6287 do { 6288 this.urlLength = cast(int) url.length; 6289 this.typeLength = cast(int) type.length; 6290 this.messageLength = cast(int) message.length; 6291 this._lifetime = lifetime; 6292 6293 this.urlBuffer[0 .. url.length] = url[]; 6294 this.typeBuffer[0 .. type.length] = type[]; 6295 this.messageBuffer[0 .. message.length] = message[]; 6296 } 6297 } 6298 6299 struct EventConnection { 6300 int fd; 6301 bool needsChunking; 6302 } 6303 6304 private EventConnection[][string] eventConnectionsByUrl; 6305 private string[][string] pipes; 6306 6307 private void handleInputEvent(scope SendableEvent* event) { 6308 static int eventId; 6309 6310 static struct StoredEvent { 6311 int id; 6312 string type; 6313 string message; 6314 int lifetimeRemaining; 6315 } 6316 6317 StoredEvent[][string] byUrl; 6318 6319 int thisId = ++eventId; 6320 6321 if(event.lifetime) 6322 byUrl[event.url.idup] ~= StoredEvent(thisId, event.type.idup, event.message.idup, event.lifetime); 6323 6324 auto connectionsPtr = event.url in eventConnectionsByUrl; 6325 EventConnection[] connections; 6326 if(connectionsPtr is null) 6327 return; 6328 else 6329 connections = *connectionsPtr; 6330 6331 char[4096] buffer; 6332 char[] formattedMessage; 6333 6334 void append(const char[] a) { 6335 // the 6's here are to leave room for a HTTP chunk header, if it proves necessary 6336 buffer[6 + formattedMessage.length .. 6 + formattedMessage.length + a.length] = a[]; 6337 formattedMessage = buffer[6 .. 6 + formattedMessage.length + a.length]; 6338 } 6339 6340 import std.algorithm.iteration; 6341 6342 if(connections.length) { 6343 append("id: "); 6344 append(to!string(thisId)); 6345 append("\n"); 6346 6347 append("event: "); 6348 append(event.type); 6349 append("\n"); 6350 6351 foreach(line; event.message.splitter("\n")) { 6352 append("data: "); 6353 append(line); 6354 append("\n"); 6355 } 6356 6357 append("\n"); 6358 } 6359 6360 // chunk it for HTTP! 6361 auto len = toHex(formattedMessage.length); 6362 buffer[4 .. 6] = "\r\n"[]; 6363 buffer[4 - len.length .. 4] = len[]; 6364 6365 auto chunkedMessage = buffer[4 - len.length .. 6 + formattedMessage.length]; 6366 // done 6367 6368 // FIXME: send back requests when needed 6369 // FIXME: send a single ":\n" every 15 seconds to keep alive 6370 6371 foreach(connection; connections) { 6372 if(connection.needsChunking) 6373 nonBlockingWrite(this, connection.fd, chunkedMessage); 6374 else 6375 nonBlockingWrite(this, connection.fd, formattedMessage); 6376 } 6377 } 6378 } 6379 6380 void runAddonServer(EIS)(string localListenerName, EIS eis) if(is(EIS : EventIoServer)) { 6381 version(Posix) { 6382 6383 import core.sys.posix.unistd; 6384 import core.sys.posix.fcntl; 6385 import core.sys.posix.sys.un; 6386 6387 import core.sys.posix.signal; 6388 signal(SIGPIPE, SIG_IGN); 6389 6390 int sock = socket(AF_UNIX, SOCK_STREAM, 0); 6391 if(sock == -1) 6392 throw new Exception("socket " ~ to!string(errno)); 6393 6394 scope(failure) 6395 close(sock); 6396 6397 // add-on server processes are assumed to be local, and thus will 6398 // use unix domain sockets. Besides, I want to pass sockets to them, 6399 // so it basically must be local (except for the session server, but meh). 6400 sockaddr_un addr; 6401 addr.sun_family = AF_UNIX; 6402 version(linux) { 6403 // on linux, we will use the abstract namespace 6404 addr.sun_path[0] = 0; 6405 addr.sun_path[1 .. localListenerName.length + 1] = cast(typeof(addr.sun_path[])) localListenerName[]; 6406 } else { 6407 // but otherwise, just use a file cuz we must. 6408 addr.sun_path[0 .. localListenerName.length] = cast(typeof(addr.sun_path[])) localListenerName[]; 6409 } 6410 6411 if(bind(sock, cast(sockaddr*) &addr, addr.sizeof) == -1) 6412 throw new Exception("bind " ~ to!string(errno)); 6413 6414 if(listen(sock, 128) == -1) 6415 throw new Exception("listen " ~ to!string(errno)); 6416 6417 makeNonBlocking(sock); 6418 6419 version(linux) { 6420 import core.sys.linux.epoll; 6421 auto epoll_fd = epoll_create1(EPOLL_CLOEXEC); 6422 if(epoll_fd == -1) 6423 throw new Exception("epoll_create1 " ~ to!string(errno)); 6424 scope(failure) 6425 close(epoll_fd); 6426 } else { 6427 import core.sys.posix.poll; 6428 } 6429 6430 eis.epoll_fd = epoll_fd; 6431 6432 auto acceptOp = allocateIoOp(sock, IoOp.Read, 0, null); 6433 scope(exit) 6434 freeIoOp(acceptOp); 6435 6436 version(linux) { 6437 epoll_event ev; 6438 ev.events = EPOLLIN | EPOLLET; 6439 ev.data.ptr = acceptOp; 6440 if(epoll_ctl(epoll_fd, EPOLL_CTL_ADD, sock, &ev) == -1) 6441 throw new Exception("epoll_ctl " ~ to!string(errno)); 6442 6443 epoll_event[64] events; 6444 } else { 6445 pollfd[] pollfds; 6446 IoOp*[int] ioops; 6447 pollfds ~= pollfd(sock, POLLIN); 6448 ioops[sock] = acceptOp; 6449 } 6450 6451 while(true) { 6452 6453 // FIXME: it should actually do a timerfd that runs on any thing that hasn't been run recently 6454 6455 int timeout_milliseconds = 15000; // -1; // infinite 6456 //writeln("waiting for ", name); 6457 6458 version(linux) { 6459 auto nfds = epoll_wait(epoll_fd, events.ptr, events.length, timeout_milliseconds); 6460 if(nfds == -1) { 6461 if(errno == EINTR) 6462 continue; 6463 throw new Exception("epoll_wait " ~ to!string(errno)); 6464 } 6465 } else { 6466 int nfds = poll(pollfds.ptr, cast(int) pollfds.length, timeout_milliseconds); 6467 size_t lastIdx = 0; 6468 } 6469 6470 if(nfds == 0) { 6471 eis.wait_timeout(); 6472 } 6473 6474 foreach(idx; 0 .. nfds) { 6475 version(linux) { 6476 auto flags = events[idx].events; 6477 auto ioop = cast(IoOp*) events[idx].data.ptr; 6478 } else { 6479 IoOp* ioop; 6480 foreach(tidx, thing; pollfds[lastIdx .. $]) { 6481 if(thing.revents) { 6482 ioop = ioops[thing.fd]; 6483 lastIdx += tidx + 1; 6484 break; 6485 } 6486 } 6487 } 6488 6489 //writeln(flags, " ", ioop.fd); 6490 6491 void newConnection() { 6492 // on edge triggering, it is important that we get it all 6493 while(true) { 6494 auto size = cast(uint) addr.sizeof; 6495 auto ns = accept(sock, cast(sockaddr*) &addr, &size); 6496 if(ns == -1) { 6497 if(errno == EAGAIN || errno == EWOULDBLOCK) { 6498 // all done, got it all 6499 break; 6500 } 6501 throw new Exception("accept " ~ to!string(errno)); 6502 } 6503 6504 makeNonBlocking(ns); 6505 auto niop = allocateIoOp(ns, IoOp.ReadSocketHandle, 4096, &eis.handleLocalConnectionData); 6506 niop.closeHandler = &eis.handleLocalConnectionClose; 6507 niop.completeHandler = &eis.handleLocalConnectionComplete; 6508 scope(failure) freeIoOp(niop); 6509 6510 version(linux) { 6511 epoll_event nev; 6512 nev.events = EPOLLIN | EPOLLET; 6513 nev.data.ptr = niop; 6514 if(epoll_ctl(epoll_fd, EPOLL_CTL_ADD, ns, &nev) == -1) 6515 throw new Exception("epoll_ctl " ~ to!string(errno)); 6516 } else { 6517 bool found = false; 6518 foreach(ref pfd; pollfds) { 6519 if(pfd.fd < 0) { 6520 pfd.fd = ns; 6521 found = true; 6522 } 6523 } 6524 if(!found) 6525 pollfds ~= pollfd(ns, POLLIN); 6526 ioops[ns] = niop; 6527 } 6528 } 6529 } 6530 6531 bool newConnectionCondition() { 6532 version(linux) 6533 return ioop.fd == sock && (flags & EPOLLIN); 6534 else 6535 return pollfds[idx].fd == sock && (pollfds[idx].revents & POLLIN); 6536 } 6537 6538 if(newConnectionCondition()) { 6539 newConnection(); 6540 } else if(ioop.operation == IoOp.ReadSocketHandle) { 6541 while(true) { 6542 int in_fd; 6543 auto got = read_fd(ioop.fd, ioop.allocatedBuffer.ptr, ioop.allocatedBuffer.length, &in_fd); 6544 if(got == -1) { 6545 if(errno == EAGAIN || errno == EWOULDBLOCK) { 6546 // all done, got it all 6547 if(ioop.completeHandler) 6548 ioop.completeHandler(ioop); 6549 break; 6550 } 6551 throw new Exception("recv " ~ to!string(errno)); 6552 } 6553 6554 if(got == 0) { 6555 if(ioop.closeHandler) { 6556 ioop.closeHandler(ioop); 6557 version(linux) {} // nothing needed 6558 else { 6559 foreach(ref pfd; pollfds) { 6560 if(pfd.fd == ioop.fd) 6561 pfd.fd = -1; 6562 } 6563 } 6564 } 6565 close(ioop.fd); 6566 freeIoOp(ioop); 6567 break; 6568 } 6569 6570 ioop.bufferLengthUsed = cast(int) got; 6571 ioop.handler(ioop, in_fd); 6572 } 6573 } else if(ioop.operation == IoOp.Read) { 6574 while(true) { 6575 auto got = read(ioop.fd, ioop.allocatedBuffer.ptr, ioop.allocatedBuffer.length); 6576 if(got == -1) { 6577 if(errno == EAGAIN || errno == EWOULDBLOCK) { 6578 // all done, got it all 6579 if(ioop.completeHandler) 6580 ioop.completeHandler(ioop); 6581 break; 6582 } 6583 throw new Exception("recv " ~ to!string(ioop.fd) ~ " errno " ~ to!string(errno)); 6584 } 6585 6586 if(got == 0) { 6587 if(ioop.closeHandler) 6588 ioop.closeHandler(ioop); 6589 close(ioop.fd); 6590 freeIoOp(ioop); 6591 break; 6592 } 6593 6594 ioop.bufferLengthUsed = cast(int) got; 6595 if(ioop.handler(ioop, ioop.fd)) { 6596 close(ioop.fd); 6597 freeIoOp(ioop); 6598 break; 6599 } 6600 } 6601 } 6602 6603 // EPOLLHUP? 6604 } 6605 } 6606 } else version(Windows) { 6607 6608 // set up a named pipe 6609 // https://msdn.microsoft.com/en-us/library/windows/desktop/ms724251(v=vs.85).aspx 6610 // https://docs.microsoft.com/en-us/windows/desktop/api/winsock2/nf-winsock2-wsaduplicatesocketw 6611 // https://docs.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-getnamedpipeserverprocessid 6612 6613 } else static assert(0); 6614 } 6615 6616 6617 version(with_sendfd) 6618 // copied from the web and ported from C 6619 // see https://stackoverflow.com/questions/2358684/can-i-share-a-file-descriptor-to-another-process-on-linux-or-are-they-local-to-t 6620 ssize_t write_fd(int fd, void *ptr, size_t nbytes, int sendfd) { 6621 msghdr msg; 6622 iovec[1] iov; 6623 6624 version(OSX) { 6625 //msg.msg_accrights = cast(cattr_t) &sendfd; 6626 //msg.msg_accrightslen = int.sizeof; 6627 } else { 6628 union ControlUnion { 6629 cmsghdr cm; 6630 char[CMSG_SPACE(int.sizeof)] control; 6631 } 6632 6633 ControlUnion control_un; 6634 cmsghdr* cmptr; 6635 6636 msg.msg_control = control_un.control.ptr; 6637 msg.msg_controllen = control_un.control.length; 6638 6639 cmptr = CMSG_FIRSTHDR(&msg); 6640 cmptr.cmsg_len = CMSG_LEN(int.sizeof); 6641 cmptr.cmsg_level = SOL_SOCKET; 6642 cmptr.cmsg_type = SCM_RIGHTS; 6643 *(cast(int *) CMSG_DATA(cmptr)) = sendfd; 6644 } 6645 6646 msg.msg_name = null; 6647 msg.msg_namelen = 0; 6648 6649 iov[0].iov_base = ptr; 6650 iov[0].iov_len = nbytes; 6651 msg.msg_iov = iov.ptr; 6652 msg.msg_iovlen = 1; 6653 6654 return sendmsg(fd, &msg, 0); 6655 } 6656 6657 version(with_sendfd) 6658 // copied from the web and ported from C 6659 ssize_t read_fd(int fd, void *ptr, size_t nbytes, int *recvfd) { 6660 msghdr msg; 6661 iovec[1] iov; 6662 ssize_t n; 6663 int newfd; 6664 6665 version(OSX) { 6666 //msg.msg_accrights = cast(cattr_t) recvfd; 6667 //msg.msg_accrightslen = int.sizeof; 6668 } else { 6669 union ControlUnion { 6670 cmsghdr cm; 6671 char[CMSG_SPACE(int.sizeof)] control; 6672 } 6673 ControlUnion control_un; 6674 cmsghdr* cmptr; 6675 6676 msg.msg_control = control_un.control.ptr; 6677 msg.msg_controllen = control_un.control.length; 6678 } 6679 6680 msg.msg_name = null; 6681 msg.msg_namelen = 0; 6682 6683 iov[0].iov_base = ptr; 6684 iov[0].iov_len = nbytes; 6685 msg.msg_iov = iov.ptr; 6686 msg.msg_iovlen = 1; 6687 6688 if ( (n = recvmsg(fd, &msg, 0)) <= 0) 6689 return n; 6690 6691 version(OSX) { 6692 //if(msg.msg_accrightslen != int.sizeof) 6693 //*recvfd = -1; 6694 } else { 6695 if ( (cmptr = CMSG_FIRSTHDR(&msg)) != null && 6696 cmptr.cmsg_len == CMSG_LEN(int.sizeof)) { 6697 if (cmptr.cmsg_level != SOL_SOCKET) 6698 throw new Exception("control level != SOL_SOCKET"); 6699 if (cmptr.cmsg_type != SCM_RIGHTS) 6700 throw new Exception("control type != SCM_RIGHTS"); 6701 *recvfd = *(cast(int *) CMSG_DATA(cmptr)); 6702 } else 6703 *recvfd = -1; /* descriptor was not passed */ 6704 } 6705 6706 return n; 6707 } 6708 /* end read_fd */ 6709 6710 6711 /* 6712 Event source stuff 6713 6714 The api is: 6715 6716 sendEvent(string url, string type, string data, int timeout = 60*10); 6717 6718 attachEventListener(string url, int fd, lastId) 6719 6720 6721 It just sends to all attached listeners, and stores it until the timeout 6722 for replaying via lastEventId. 6723 */ 6724 6725 /* 6726 Session process stuff 6727 6728 it stores it all. the cgi object has a session object that can grab it 6729 6730 session may be done in the same process if possible, there is a version 6731 switch to choose if you want to override. 6732 */ 6733 6734 struct DispatcherDefinition(alias dispatchHandler, DispatcherDetails = typeof(null)) {// if(is(typeof(dispatchHandler("str", Cgi.init, void) == bool))) { // bool delegate(string urlPrefix, Cgi cgi) dispatchHandler; 6735 alias handler = dispatchHandler; 6736 string urlPrefix; 6737 bool rejectFurther; 6738 immutable(DispatcherDetails) details; 6739 } 6740 6741 private string urlify(string name) { 6742 return beautify(name, '-', true); 6743 } 6744 6745 private string beautify(string name, char space = ' ', bool allLowerCase = false) { 6746 if(name == "id") 6747 return allLowerCase ? name : "ID"; 6748 6749 char[160] buffer; 6750 int bufferIndex = 0; 6751 bool shouldCap = true; 6752 bool shouldSpace; 6753 bool lastWasCap; 6754 foreach(idx, char ch; name) { 6755 if(bufferIndex == buffer.length) return name; // out of space, just give up, not that important 6756 6757 if((ch >= 'A' && ch <= 'Z') || ch == '_') { 6758 if(lastWasCap) { 6759 // two caps in a row, don't change. Prolly acronym. 6760 } else { 6761 if(idx) 6762 shouldSpace = true; // new word, add space 6763 } 6764 6765 lastWasCap = true; 6766 } else { 6767 lastWasCap = false; 6768 } 6769 6770 if(shouldSpace) { 6771 buffer[bufferIndex++] = space; 6772 if(bufferIndex == buffer.length) return name; // out of space, just give up, not that important 6773 shouldSpace = false; 6774 } 6775 if(shouldCap) { 6776 if(ch >= 'a' && ch <= 'z') 6777 ch -= 32; 6778 shouldCap = false; 6779 } 6780 if(allLowerCase && ch >= 'A' && ch <= 'Z') 6781 ch += 32; 6782 buffer[bufferIndex++] = ch; 6783 } 6784 return buffer[0 .. bufferIndex].idup; 6785 } 6786 6787 /* 6788 string urlFor(alias func)() { 6789 return __traits(identifier, func); 6790 } 6791 */ 6792 6793 /++ 6794 UDA: The name displayed to the user in auto-generated HTML. 6795 6796 Default is `beautify(identifier)`. 6797 +/ 6798 struct DisplayName { 6799 string name; 6800 } 6801 6802 /++ 6803 UDA: The name used in the URL or web parameter. 6804 6805 Default is `urlify(identifier)` for functions and `identifier` for parameters and data members. 6806 +/ 6807 struct UrlName { 6808 string name; 6809 } 6810 6811 /++ 6812 UDA: default format to respond for this method 6813 +/ 6814 struct DefaultFormat { string value; } 6815 6816 class MissingArgumentException : Exception { 6817 string functionName; 6818 string argumentName; 6819 string argumentType; 6820 6821 this(string functionName, string argumentName, string argumentType, string file = __FILE__, size_t line = __LINE__, Throwable next = null) { 6822 this.functionName = functionName; 6823 this.argumentName = argumentName; 6824 this.argumentType = argumentType; 6825 6826 super("Missing Argument: " ~ this.argumentName, file, line, next); 6827 } 6828 } 6829 6830 /++ 6831 This can be attached to any constructor or function called from the cgi system. 6832 6833 If it is present, the function argument can NOT be set from web params, but instead 6834 is set to the return value of the given `func`. 6835 6836 If `func` can take a parameter of type [Cgi], it will be passed the one representing 6837 the current request. Otherwise, it must take zero arguments. 6838 6839 Any params in your function of type `Cgi` are automatically assumed to take the cgi object 6840 for the connection. Any of type [Session] (with an argument) is also assumed to come from 6841 the cgi object. 6842 6843 const arguments are also supported. 6844 +/ 6845 struct ifCalledFromWeb(alias func) {} 6846 6847 // it only looks at query params for GET requests, the rest must be in the body for a function argument. 6848 auto callFromCgi(alias method, T)(T dg, Cgi cgi) { 6849 6850 // FIXME: any array of structs should also be settable or gettable from csv as well. 6851 6852 // FIXME: think more about checkboxes and bools. 6853 6854 import std.traits; 6855 6856 Parameters!method params; 6857 alias idents = ParameterIdentifierTuple!method; 6858 alias defaults = ParameterDefaults!method; 6859 6860 const(string)[] names; 6861 const(string)[] values; 6862 6863 // first, check for missing arguments and initialize to defaults if necessary 6864 6865 static if(is(typeof(method) P == __parameters)) 6866 static foreach(idx, param; P) {{ 6867 // see: mustNotBeSetFromWebParams 6868 static if(is(param : Cgi)) { 6869 static assert(!is(param == immutable)); 6870 cast() params[idx] = cgi; 6871 } else static if(is(param == Session!D, D)) { 6872 static assert(!is(param == immutable)); 6873 cast() params[idx] = cgi.getSessionObject!D(); 6874 } else { 6875 bool populated; 6876 static foreach(uda; __traits(getAttributes, P[idx .. idx + 1])) { 6877 static if(is(uda == ifCalledFromWeb!func, alias func)) { 6878 static if(is(typeof(func(cgi)))) 6879 params[idx] = func(cgi); 6880 else 6881 params[idx] = func(); 6882 6883 populated = true; 6884 } 6885 } 6886 6887 if(!populated) { 6888 static if(__traits(compiles, { params[idx] = param.getAutomaticallyForCgi(cgi); } )) { 6889 params[idx] = param.getAutomaticallyForCgi(cgi); 6890 populated = true; 6891 } 6892 } 6893 6894 if(!populated) { 6895 auto ident = idents[idx]; 6896 if(cgi.requestMethod == Cgi.RequestMethod.GET) { 6897 if(ident !in cgi.get) { 6898 static if(is(defaults[idx] == void)) { 6899 static if(is(param == bool)) 6900 params[idx] = false; 6901 else 6902 throw new MissingArgumentException(__traits(identifier, method), ident, param.stringof); 6903 } else 6904 params[idx] = defaults[idx]; 6905 } 6906 } else { 6907 if(ident !in cgi.post) { 6908 static if(is(defaults[idx] == void)) { 6909 static if(is(param == bool)) 6910 params[idx] = false; 6911 else 6912 throw new MissingArgumentException(__traits(identifier, method), ident, param.stringof); 6913 } else 6914 params[idx] = defaults[idx]; 6915 } 6916 } 6917 } 6918 } 6919 }} 6920 6921 // second, parse the arguments in order to build up arrays, etc. 6922 6923 static bool setVariable(T)(string name, string paramName, T* what, string value) { 6924 static if(is(T == struct)) { 6925 if(name == paramName) { 6926 *what = T.init; 6927 return true; 6928 } else { 6929 // could be a child 6930 if(name[paramName.length] == '.') { 6931 paramName = name[paramName.length + 1 .. $]; 6932 name = paramName; 6933 int p = 0; 6934 foreach(ch; paramName) { 6935 if(ch == '.' || ch == '[') 6936 break; 6937 p++; 6938 } 6939 6940 // set the child member 6941 switch(paramName) { 6942 static foreach(idx, memberName; __traits(allMembers, T)) 6943 static if(__traits(compiles, __traits(getMember, T, memberName).offsetof)) { 6944 // data member! 6945 case memberName: 6946 return setVariable(name, paramName, &(__traits(getMember, *what, memberName)), value); 6947 } 6948 default: 6949 // ok, not a member 6950 } 6951 } 6952 } 6953 6954 return false; 6955 } else static if(isSomeString!T || isIntegral!T || isFloatingPoint!T) { 6956 *what = to!T(value); 6957 return true; 6958 } else static if(is(T == bool)) { 6959 *what = value == "1" || value == "yes" || value == "t" || value == "true" || value == "on"; 6960 return true; 6961 } else static if(is(T == K[], K)) { 6962 K tmp; 6963 if(name == paramName) { 6964 // direct - set and append 6965 if(setVariable(name, paramName, &tmp, value)) { 6966 (*what) ~= tmp; 6967 return true; 6968 } else { 6969 return false; 6970 } 6971 } else { 6972 // child, append to last element 6973 // FIXME: what about range violations??? 6974 auto ptr = &(*what)[(*what).length - 1]; 6975 return setVariable(name, paramName, ptr, value); 6976 6977 } 6978 } else static if(is(T == V[K], K, V)) { 6979 // assoc array, name[key] is valid 6980 if(name == paramName) { 6981 // no action necessary 6982 return true; 6983 } else if(name[paramName.length] == '[') { 6984 int count = 1; 6985 auto idx = paramName.length + 1; 6986 while(idx < name.length && count > 0) { 6987 if(name[idx] == '[') 6988 count++; 6989 else if(name[idx] == ']') { 6990 count--; 6991 if(count == 0) break; 6992 } 6993 idx++; 6994 } 6995 if(idx == name.length) 6996 return false; // malformed 6997 6998 auto insideBrackets = name[paramName.length + 1 .. idx]; 6999 auto afterName = name[idx + 1 .. $]; 7000 7001 auto k = to!K(insideBrackets); 7002 V v; 7003 if(auto ptr = k in *what) 7004 v = *ptr; 7005 7006 name = name[0 .. paramName.length]; 7007 //writeln(name, afterName, " ", paramName); 7008 7009 auto ret = setVariable(name ~ afterName, paramName, &v, value); 7010 if(ret) { 7011 (*what)[k] = v; 7012 return true; 7013 } 7014 } 7015 7016 return false; 7017 } else { 7018 static assert(0, "unsupported type for cgi call " ~ T.stringof); 7019 } 7020 7021 //return false; 7022 } 7023 7024 void setArgument(string name, string value) { 7025 int p; 7026 foreach(ch; name) { 7027 if(ch == '.' || ch == '[') 7028 break; 7029 p++; 7030 } 7031 7032 auto paramName = name[0 .. p]; 7033 7034 sw: switch(paramName) { 7035 static if(is(typeof(method) P == __parameters)) 7036 static foreach(idx, param; P) { 7037 static if(mustNotBeSetFromWebParams!(P[idx], __traits(getAttributes, P[idx .. idx + 1]))) { 7038 // cannot be set from the outside 7039 } else { 7040 case idents[idx]: 7041 static if(is(param == Cgi.UploadedFile)) { 7042 params[idx] = cgi.files[name]; 7043 } else { 7044 setVariable(name, paramName, ¶ms[idx], value); 7045 } 7046 break sw; 7047 } 7048 } 7049 default: 7050 // ignore; not relevant argument 7051 } 7052 } 7053 7054 if(cgi.requestMethod == Cgi.RequestMethod.GET) { 7055 names = cgi.allGetNamesInOrder; 7056 values = cgi.allGetValuesInOrder; 7057 } else { 7058 names = cgi.allPostNamesInOrder; 7059 values = cgi.allPostValuesInOrder; 7060 } 7061 7062 foreach(idx, name; names) { 7063 setArgument(name, values[idx]); 7064 } 7065 7066 static if(is(ReturnType!method == void)) { 7067 typeof(null) ret; 7068 dg(params); 7069 } else { 7070 auto ret = dg(params); 7071 } 7072 7073 // FIXME: format return values 7074 // options are: json, html, csv. 7075 // also may need to wrap in envelope format: none, html, or json. 7076 return ret; 7077 } 7078 7079 private bool mustNotBeSetFromWebParams(T, attrs...)() { 7080 static if(is(T : const(Cgi))) { 7081 return true; 7082 } else static if(is(T : const(Session!D), D)) { 7083 return true; 7084 } else static if(__traits(compiles, T.getAutomaticallyForCgi(Cgi.init))) { 7085 return true; 7086 } else { 7087 static foreach(uda; attrs) 7088 static if(is(uda == ifCalledFromWeb!func, alias func)) 7089 return true; 7090 return false; 7091 } 7092 } 7093 7094 private bool hasIfCalledFromWeb(attrs...)() { 7095 static foreach(uda; attrs) 7096 static if(is(uda == ifCalledFromWeb!func, alias func)) 7097 return true; 7098 return false; 7099 } 7100 7101 /+ 7102 Argument conversions: for the most part, it is to!Thing(string). 7103 7104 But arrays and structs are a bit different. Arrays come from the cgi array. Thus 7105 they are passed 7106 7107 arr=foo&arr=bar <-- notice the same name. 7108 7109 Structs are first declared with an empty thing, then have their members set individually, 7110 with dot notation. The members are not required, just the initial declaration. 7111 7112 struct Foo { 7113 int a; 7114 string b; 7115 } 7116 void test(Foo foo){} 7117 7118 foo&foo.a=5&foo.b=str <-- the first foo declares the arg, the others set the members 7119 7120 Arrays of structs use this declaration. 7121 7122 void test(Foo[] foo) {} 7123 7124 foo&foo.a=5&foo.b=bar&foo&foo.a=9 7125 7126 You can use a hidden input field in HTML forms to achieve this. The value of the naked name 7127 declaration is ignored. 7128 7129 Mind that order matters! The declaration MUST come first in the string. 7130 7131 Arrays of struct members follow this rule recursively. 7132 7133 struct Foo { 7134 int[] a; 7135 } 7136 7137 foo&foo.a=1&foo.a=2&foo&foo.a=1 7138 7139 7140 Associative arrays are formatted with brackets, after a declaration, like structs: 7141 7142 foo&foo[key]=value&foo[other_key]=value 7143 7144 7145 Note: for maximum compatibility with outside code, keep your types simple. Some libraries 7146 do not support the strict ordering requirements to work with these struct protocols. 7147 7148 FIXME: also perhaps accept application/json to better work with outside trash. 7149 7150 7151 Return values are also auto-formatted according to user-requested type: 7152 for json, it loops over and converts. 7153 for html, basic types are strings. Arrays are <ol>. Structs are <dl>. Arrays of structs are tables! 7154 +/ 7155 7156 /++ 7157 A web presenter is responsible for rendering things to HTML to be usable 7158 in a web browser. 7159 7160 They are passed as template arguments to the base classes of [WebObject] 7161 7162 Responsible for displaying stuff as HTML. You can put this into your own aggregate 7163 and override it. Use forwarding and specialization to customize it. 7164 7165 When you inherit from it, pass your own class as the CRTP argument. This lets the base 7166 class templates and your overridden templates work with each other. 7167 7168 --- 7169 class MyPresenter : WebPresenter!(MyPresenter) { 7170 @Override 7171 void presentSuccessfulReturnAsHtml(T : CustomType)(Cgi cgi, T ret) { 7172 // present the CustomType 7173 } 7174 @Override 7175 void presentSuccessfulReturnAsHtml(T)(Cgi cgi, T ret) { 7176 // handle everything else via the super class, which will call 7177 // back to your class when appropriate 7178 super.presentSuccessfulReturnAsHtml(cgi, ret); 7179 } 7180 } 7181 --- 7182 7183 +/ 7184 class WebPresenter(CRTP) { 7185 7186 /// A UDA version of the built-in `override`, to be used for static template polymorphism 7187 /// If you override a plain method, use `override`. If a template, use `@Override`. 7188 enum Override; 7189 7190 string script() { 7191 return ` 7192 `; 7193 } 7194 7195 string style() { 7196 return ` 7197 :root { 7198 --mild-border: #ccc; 7199 --middle-border: #999; 7200 --accent-color: #e8e8e8; 7201 --sidebar-color: #f2f2f2; 7202 } 7203 ` ~ genericFormStyling() ~ genericSiteStyling(); 7204 } 7205 7206 string genericFormStyling() { 7207 return 7208 q"css 7209 table.automatic-data-display { 7210 border-collapse: collapse; 7211 border: solid 1px var(--mild-border); 7212 } 7213 7214 table.automatic-data-display td { 7215 vertical-align: top; 7216 border: solid 1px var(--mild-border); 7217 padding: 2px 4px; 7218 } 7219 7220 table.automatic-data-display th { 7221 border: solid 1px var(--mild-border); 7222 border-bottom: solid 1px var(--middle-border); 7223 padding: 2px 4px; 7224 } 7225 7226 ol.automatic-data-display { 7227 margin: 0px; 7228 list-style-position: inside; 7229 padding: 0px; 7230 } 7231 7232 .automatic-form { 7233 max-width: 600px; 7234 } 7235 7236 .form-field { 7237 margin: 0.5em; 7238 padding-left: 0.5em; 7239 } 7240 7241 .label-text { 7242 display: block; 7243 font-weight: bold; 7244 margin-left: -0.5em; 7245 } 7246 7247 .add-array-button { 7248 7249 } 7250 css"; 7251 } 7252 7253 string genericSiteStyling() { 7254 return 7255 q"css 7256 * { box-sizing: border-box; } 7257 html, body { margin: 0px; } 7258 body { 7259 font-family: sans-serif; 7260 } 7261 header { 7262 background: var(--accent-color); 7263 height: 64px; 7264 } 7265 footer { 7266 background: var(--accent-color); 7267 height: 64px; 7268 } 7269 #site-container { 7270 display: flex; 7271 } 7272 main { 7273 flex: 1 1 auto; 7274 order: 2; 7275 min-height: calc(100vh - 64px - 64px); 7276 padding: 4px; 7277 padding-left: 1em; 7278 } 7279 #sidebar { 7280 flex: 0 0 16em; 7281 order: 1; 7282 background: var(--sidebar-color); 7283 } 7284 css"; 7285 } 7286 7287 import arsd.dom; 7288 Element htmlContainer() { 7289 auto document = new Document(q"html 7290 <!DOCTYPE html> 7291 <html> 7292 <head> 7293 <title>D Application</title> 7294 <link rel="stylesheet" href="style.css" /> 7295 </head> 7296 <body> 7297 <header></header> 7298 <div id="site-container"> 7299 <main></main> 7300 <div id="sidebar"></div> 7301 </div> 7302 <footer></footer> 7303 <script src="script.js"></script> 7304 </body> 7305 </html> 7306 html", true, true); 7307 7308 return document.requireSelector("main"); 7309 } 7310 7311 /// Renders a response as an HTTP error 7312 void renderBasicError(Cgi cgi, int httpErrorCode) { 7313 cgi.setResponseStatus(getHttpCodeText(httpErrorCode)); 7314 auto c = htmlContainer(); 7315 c.innerText = getHttpCodeText(httpErrorCode); 7316 cgi.setResponseContentType("text/html; charset=utf-8"); 7317 cgi.write(c.parentDocument.toString(), true); 7318 } 7319 7320 /// typeof(null) (which is also used to represent functions returning `void`) do nothing 7321 /// in the default presenter - allowing the function to have full low-level control over the 7322 /// response. 7323 void presentSuccessfulReturnAsHtml(T : typeof(null))(Cgi cgi, T ret) { 7324 // nothing intentionally! 7325 } 7326 7327 /// Redirections are forwarded to [Cgi.setResponseLocation] 7328 void presentSuccessfulReturnAsHtml(T : Redirection)(Cgi cgi, T ret) { 7329 cgi.setResponseLocation(ret.to, true, getHttpCodeText(ret.code)); 7330 } 7331 7332 /// Multiple responses deconstruct the algebraic type and forward to the appropriate handler at runtime 7333 void presentSuccessfulReturnAsHtml(T : MultipleResponses!Types, Types...)(Cgi cgi, T ret) { 7334 bool outputted = false; 7335 static foreach(index, type; Types) { 7336 if(ret.contains == index) { 7337 assert(!outputted); 7338 outputted = true; 7339 (cast(CRTP) this).presentSuccessfulReturnAsHtml(cgi, ret.payload[index]); 7340 } 7341 } 7342 if(!outputted) 7343 assert(0); 7344 } 7345 7346 /// An instance of the [arsd.dom.FileResource] interface has its own content type; assume it is a download of some sort. 7347 void presentSuccessfulReturnAsHtml(T : FileResource)(Cgi cgi, T ret) { 7348 cgi.setCache(true); // not necessarily true but meh 7349 cgi.setResponseContentType(ret.contentType); 7350 cgi.write(ret.getData(), true); 7351 } 7352 7353 /// And the default handler will call [formatReturnValueAsHtml] and place it inside the [htmlContainer]. 7354 void presentSuccessfulReturnAsHtml(T)(Cgi cgi, T ret) { 7355 auto container = this.htmlContainer(); 7356 container.appendChild(formatReturnValueAsHtml(ret)); 7357 cgi.write(container.parentDocument.toString(), true); 7358 } 7359 7360 /++ 7361 If you override this, you will need to cast the exception type `t` dynamically, 7362 but can then use the template arguments here to refer back to the function. 7363 7364 `func` is an alias to the method itself, and `dg` is a callable delegate to the same 7365 method on the live object. You could, in theory, change arguments and retry, but I 7366 provide that information mostly with the expectation that you will use them to make 7367 useful forms or richer error messages for the user. 7368 +/ 7369 void presentExceptionAsHtml(alias func, T)(Cgi cgi, Throwable t, T dg) { 7370 if(auto mae = cast(MissingArgumentException) t) { 7371 auto container = this.htmlContainer(); 7372 container.appendChild(Element.make("p", "Argument `" ~ mae.argumentName ~ "` of type `" ~ mae.argumentType ~ "` is missing")); 7373 container.appendChild(createAutomaticFormForFunction!(func)(dg)); 7374 7375 cgi.write(container.parentDocument.toString(), true); 7376 } else { 7377 auto container = this.htmlContainer(); 7378 7379 // import std.stdio; writeln(t.toString()); 7380 7381 container.appendChild(exceptionToElement(t)); 7382 7383 container.addChild("h4", "GET"); 7384 foreach(k, v; cgi.get) { 7385 auto deets = container.addChild("details"); 7386 deets.addChild("summary", k); 7387 deets.addChild("div", v); 7388 } 7389 7390 container.addChild("h4", "POST"); 7391 foreach(k, v; cgi.post) { 7392 auto deets = container.addChild("details"); 7393 deets.addChild("summary", k); 7394 deets.addChild("div", v); 7395 } 7396 7397 7398 if(!cgi.outputtedResponseData) 7399 cgi.setResponseStatus("500 Internal Server Error"); 7400 cgi.write(container.parentDocument.toString(), true); 7401 } 7402 } 7403 7404 Element exceptionToElement(Throwable t) { 7405 auto div = Element.make("div"); 7406 div.addClass("exception-display"); 7407 7408 div.addChild("p", t.msg); 7409 div.addChild("p", "Inner code origin: " ~ typeid(t).name ~ "@" ~ t.file ~ ":" ~ to!string(t.line)); 7410 7411 auto pre = div.addChild("pre"); 7412 string s; 7413 s = t.toString(); 7414 Element currentBox; 7415 bool on = false; 7416 foreach(line; s.splitLines) { 7417 if(!on && line.startsWith("-----")) 7418 on = true; 7419 if(!on) continue; 7420 if(line.indexOf("arsd/") != -1) { 7421 if(currentBox is null) { 7422 currentBox = pre.addChild("details"); 7423 currentBox.addChild("summary", "Framework code"); 7424 } 7425 currentBox.addChild("span", line ~ "\n"); 7426 } else { 7427 pre.addChild("span", line ~ "\n"); 7428 currentBox = null; 7429 } 7430 } 7431 7432 return div; 7433 } 7434 7435 /++ 7436 Returns an element for a particular type 7437 +/ 7438 Element elementFor(T)(string displayName, string name) { 7439 import std.traits; 7440 7441 auto div = Element.make("div"); 7442 div.addClass("form-field"); 7443 7444 static if(is(T == struct)) { 7445 if(displayName !is null) 7446 div.addChild("span", displayName, "label-text"); 7447 auto fieldset = div.addChild("fieldset"); 7448 fieldset.addChild("legend", beautify(T.stringof)); // FIXME 7449 fieldset.addChild("input", name); 7450 static foreach(idx, memberName; __traits(allMembers, T)) 7451 static if(__traits(compiles, __traits(getMember, T, memberName).offsetof)) { 7452 fieldset.appendChild(elementFor!(typeof(__traits(getMember, T, memberName)))(beautify(memberName), name ~ "." ~ memberName)); 7453 } 7454 } else static if(isSomeString!T || isIntegral!T || isFloatingPoint!T) { 7455 Element lbl; 7456 if(displayName !is null) { 7457 lbl = div.addChild("label"); 7458 lbl.addChild("span", displayName, "label-text"); 7459 lbl.appendText(" "); 7460 } else { 7461 lbl = div; 7462 } 7463 auto i = lbl.addChild("input", name); 7464 i.attrs.name = name; 7465 static if(isSomeString!T) 7466 i.attrs.type = "text"; 7467 else 7468 i.attrs.type = "number"; 7469 i.attrs.value = to!string(T.init); 7470 } else static if(is(T == bool)) { 7471 Element lbl; 7472 if(displayName !is null) { 7473 lbl = div.addChild("label"); 7474 lbl.addChild("span", displayName, "label-text"); 7475 lbl.appendText(" "); 7476 } else { 7477 lbl = div; 7478 } 7479 auto i = lbl.addChild("input", name); 7480 i.attrs.type = "checkbox"; 7481 i.attrs.value = "true"; 7482 i.attrs.name = name; 7483 } else static if(is(T == Cgi.UploadedFile)) { 7484 Element lbl; 7485 if(displayName !is null) { 7486 lbl = div.addChild("label"); 7487 lbl.addChild("span", displayName, "label-text"); 7488 lbl.appendText(" "); 7489 } else { 7490 lbl = div; 7491 } 7492 auto i = lbl.addChild("input", name); 7493 i.attrs.name = name; 7494 i.attrs.type = "file"; 7495 } else static if(is(T == K[], K)) { 7496 auto templ = div.addChild("template"); 7497 templ.appendChild(elementFor!(K)(null, name)); 7498 if(displayName !is null) 7499 div.addChild("span", displayName, "label-text"); 7500 auto btn = div.addChild("button"); 7501 btn.addClass("add-array-button"); 7502 btn.attrs.type = "button"; 7503 btn.innerText = "Add"; 7504 btn.attrs.onclick = q{ 7505 var a = document.importNode(this.parentNode.firstChild.content, true); 7506 this.parentNode.insertBefore(a, this); 7507 }; 7508 } else static if(is(T == V[K], K, V)) { 7509 div.innerText = "assoc array not implemented for automatic form at this time"; 7510 } else { 7511 static assert(0, "unsupported type for cgi call " ~ T.stringof); 7512 } 7513 7514 7515 return div; 7516 } 7517 7518 /// creates a form for gathering the function's arguments 7519 Form createAutomaticFormForFunction(alias method, T)(T dg) { 7520 7521 auto form = cast(Form) Element.make("form"); 7522 7523 form.addClass("automatic-form"); 7524 7525 form.addChild("h3", beautify(__traits(identifier, method))); 7526 7527 import std.traits; 7528 7529 //Parameters!method params; 7530 //alias idents = ParameterIdentifierTuple!method; 7531 //alias defaults = ParameterDefaults!method; 7532 7533 static if(is(typeof(method) P == __parameters)) 7534 static foreach(idx, _; P) {{ 7535 7536 alias param = P[idx .. idx + 1]; 7537 7538 static if(!mustNotBeSetFromWebParams!(param[0], __traits(getAttributes, param))) { 7539 string displayName = beautify(__traits(identifier, param)); 7540 static foreach(attr; __traits(getAttributes, param)) 7541 static if(is(typeof(attr) == DisplayName)) 7542 displayName = attr.name; 7543 auto i = form.appendChild(elementFor!(param)(displayName, __traits(identifier, param))); 7544 if(i.querySelector("input[type=file]") !is null) 7545 form.setAttribute("enctype", "multipart/form-data"); 7546 } 7547 }} 7548 7549 form.addChild("div", Html(`<input type="submit" value="Submit" />`), "submit-button-holder"); 7550 7551 return form; 7552 } 7553 7554 /// creates a form for gathering object members (for the REST object thing right now) 7555 Form createAutomaticFormForObject(T)(T obj) { 7556 auto form = cast(Form) Element.make("form"); 7557 7558 form.addClass("automatic-form"); 7559 7560 form.addChild("h3", beautify(__traits(identifier, T))); 7561 7562 import std.traits; 7563 7564 //Parameters!method params; 7565 //alias idents = ParameterIdentifierTuple!method; 7566 //alias defaults = ParameterDefaults!method; 7567 7568 static foreach(idx, memberName; __traits(derivedMembers, T)) {{ 7569 static if(__traits(compiles, __traits(getMember, obj, memberName).offsetof)) { 7570 string displayName = beautify(memberName); 7571 static foreach(attr; __traits(getAttributes, __traits(getMember, T, memberName))) 7572 static if(is(typeof(attr) == DisplayName)) 7573 displayName = attr.name; 7574 form.appendChild(elementFor!(typeof(__traits(getMember, T, memberName)))(displayName, memberName)); 7575 7576 form.setValue(memberName, to!string(__traits(getMember, obj, memberName))); 7577 }}} 7578 7579 form.addChild("div", Html(`<input type="submit" value="Submit" />`), "submit-button-holder"); 7580 7581 return form; 7582 } 7583 7584 /// 7585 Element formatReturnValueAsHtml(T)(T t) { 7586 import std.traits; 7587 7588 static if(is(T == typeof(null))) { 7589 return Element.make("span"); 7590 } else static if(is(T : Element)) { 7591 return t; 7592 } else static if(is(T == MultipleResponses!Types, Types...)) { 7593 static foreach(index, type; Types) { 7594 if(t.contains == index) 7595 return formatReturnValueAsHtml(t.payload[index]); 7596 } 7597 assert(0); 7598 } else static if(isIntegral!T || isSomeString!T || isFloatingPoint!T) { 7599 return Element.make("span", to!string(t), "automatic-data-display"); 7600 } else static if(is(T == V[K], K, V)) { 7601 auto dl = Element.make("dl"); 7602 dl.addClass("automatic-data-display"); 7603 foreach(k, v; t) { 7604 dl.addChild("dt", to!string(k)); 7605 dl.addChild("dd", formatReturnValueAsHtml(v)); 7606 } 7607 return dl; 7608 } else static if(is(T == struct)) { 7609 auto dl = Element.make("dl"); 7610 dl.addClass("automatic-data-display"); 7611 7612 static foreach(idx, memberName; __traits(allMembers, T)) 7613 static if(__traits(compiles, __traits(getMember, T, memberName).offsetof)) { 7614 dl.addChild("dt", memberName); 7615 dl.addChild("dt", formatReturnValueAsHtml(__traits(getMember, t, memberName))); 7616 } 7617 7618 return dl; 7619 } else static if(is(T == bool)) { 7620 return Element.make("span", t ? "true" : "false", "automatic-data-display"); 7621 } else static if(is(T == E[], E)) { 7622 static if(is(E : RestObject!Proxy, Proxy)) { 7623 // treat RestObject similar to struct 7624 auto table = cast(Table) Element.make("table"); 7625 table.addClass("automatic-data-display"); 7626 string[] names; 7627 static foreach(idx, memberName; __traits(derivedMembers, E)) 7628 static if(__traits(compiles, __traits(getMember, E, memberName).offsetof)) { 7629 names ~= beautify(memberName); 7630 } 7631 table.appendHeaderRow(names); 7632 7633 foreach(l; t) { 7634 auto tr = table.appendRow(); 7635 static foreach(idx, memberName; __traits(derivedMembers, E)) 7636 static if(__traits(compiles, __traits(getMember, E, memberName).offsetof)) { 7637 static if(memberName == "id") { 7638 string val = to!string(__traits(getMember, l, memberName)); 7639 tr.addChild("td", Element.make("a", val, E.stringof.toLower ~ "s/" ~ val)); // FIXME 7640 } else { 7641 tr.addChild("td", formatReturnValueAsHtml(__traits(getMember, l, memberName))); 7642 } 7643 } 7644 } 7645 7646 return table; 7647 } else static if(is(E == struct)) { 7648 // an array of structs is kinda special in that I like 7649 // having those formatted as tables. 7650 auto table = cast(Table) Element.make("table"); 7651 table.addClass("automatic-data-display"); 7652 string[] names; 7653 static foreach(idx, memberName; __traits(allMembers, E)) 7654 static if(__traits(compiles, __traits(getMember, E, memberName).offsetof)) { 7655 names ~= beautify(memberName); 7656 } 7657 table.appendHeaderRow(names); 7658 7659 foreach(l; t) { 7660 auto tr = table.appendRow(); 7661 static foreach(idx, memberName; __traits(allMembers, E)) 7662 static if(__traits(compiles, __traits(getMember, E, memberName).offsetof)) { 7663 tr.addChild("td", formatReturnValueAsHtml(__traits(getMember, l, memberName))); 7664 } 7665 } 7666 7667 return table; 7668 } else { 7669 // otherwise, I will just make a list. 7670 auto ol = Element.make("ol"); 7671 ol.addClass("automatic-data-display"); 7672 foreach(e; t) 7673 ol.addChild("li", formatReturnValueAsHtml(e)); 7674 return ol; 7675 } 7676 } else static assert(0, "bad return value for cgi call " ~ T.stringof); 7677 7678 assert(0); 7679 } 7680 7681 } 7682 7683 /++ 7684 The base class for the [dispatcher] function and object support. 7685 +/ 7686 class WebObject { 7687 //protected Cgi cgi; 7688 7689 protected void initialize(Cgi cgi) { 7690 //this.cgi = cgi; 7691 } 7692 } 7693 7694 /++ 7695 Can return one of the given types, decided at runtime. The syntax 7696 is to declare all the possible types in the return value, then you 7697 can `return typeof(return)(...value...)` to construct it. 7698 7699 It has an auto-generated constructor for each value it can hold. 7700 7701 --- 7702 MultipleResponses!(Redirection, string) getData(int how) { 7703 if(how & 1) 7704 return typeof(return)(Redirection("http://dpldocs.info/")); 7705 else 7706 return typeof(return)("hi there!"); 7707 } 7708 --- 7709 7710 If you have lots of returns, you could, inside the function, `alias r = typeof(return);` to shorten it a little. 7711 +/ 7712 struct MultipleResponses(T...) { 7713 private size_t contains; 7714 private union { 7715 private T payload; 7716 } 7717 7718 static foreach(index, type; T) 7719 public this(type t) { 7720 contains = index; 7721 payload[index] = t; 7722 } 7723 7724 /++ 7725 This is primarily for testing. It is your way of getting to the response. 7726 7727 Let's say you wanted to test that one holding a Redirection and a string actually 7728 holds a string, by name of "test": 7729 7730 --- 7731 auto valueToTest = your_test_function(); 7732 7733 valueToTest.visit!( 7734 (Redirection) { assert(0); }, // got a redirection instead of a string, fail the test 7735 (string s) { assert(s == "test"); } // right value, go ahead and test it. 7736 ); 7737 --- 7738 +/ 7739 void visit(Handlers...)() { 7740 template findHandler(type, HandlersToCheck...) { 7741 static if(HandlersToCheck.length == 0) 7742 alias findHandler = void; 7743 else { 7744 static if(is(typeof(HandlersToCheck[0](type.init)))) 7745 alias findHandler = handler; 7746 else 7747 alias findHandler = findHandler!(type, HandlersToCheck[1 .. $]); 7748 } 7749 } 7750 static foreach(index, type; T) {{ 7751 alias handler = findHandler!(type, Handlers); 7752 static if(is(handler == void)) 7753 static assert(0, "Type " ~ type.stringof ~ " was not handled by visitor"); 7754 else { 7755 if(index == contains) 7756 handler(payload[index]); 7757 } 7758 }} 7759 } 7760 7761 /+ 7762 auto toArsdJsvar()() { 7763 import arsd.jsvar; 7764 return var(null); 7765 } 7766 +/ 7767 } 7768 7769 struct RawResponse { 7770 int code; 7771 string[] headers; 7772 const(ubyte)[] responseBody; 7773 } 7774 7775 /++ 7776 You can return this from [WebObject] subclasses for redirections. 7777 7778 (though note the static types means that class must ALWAYS redirect if 7779 you return this directly. You might want to return [MultipleResponses] if it 7780 can be conditional) 7781 +/ 7782 struct Redirection { 7783 string to; /// The URL to redirect to. 7784 int code = 303; /// The HTTP code to retrn. 7785 } 7786 7787 /++ 7788 Serves a class' methods, as a kind of low-state RPC over the web. To be used with [dispatcher]. 7789 7790 Usage of this function will add a dependency on [arsd.dom] and [arsd.jsvar] unless you have overriden 7791 the presenter in the dispatcher. 7792 7793 FIXME: explain this better 7794 7795 You can overload functions to a limited extent: you can provide a zero-arg and non-zero-arg function, 7796 and non-zero-arg functions can filter via UDAs for various http methods. Do not attempt other overloads, 7797 the runtime result of that is undefined. 7798 7799 A method is assumed to allow any http method unless it lists some in UDAs, in which case it is limited to only those. 7800 (this might change, like maybe i will use pure as an indicator GET is ok. idk.) 7801 7802 $(WARNING 7803 --- 7804 // legal in D, undefined runtime behavior with cgi.d, it may call either method 7805 // even if you put different URL udas on it, the current code ignores them. 7806 void foo(int a) {} 7807 void foo(string a) {} 7808 --- 7809 ) 7810 7811 See_Also: [serveRestObject], [serveStaticFile] 7812 +/ 7813 auto serveApi(T)(string urlPrefix) { 7814 assert(urlPrefix[$ - 1] == '/'); 7815 return serveApiInternal!T(urlPrefix); 7816 } 7817 7818 private string nextPieceFromSlash(ref string remainingUrl) { 7819 if(remainingUrl.length == 0) 7820 return remainingUrl; 7821 int slash = 0; 7822 while(slash < remainingUrl.length && remainingUrl[slash] != '/') // && remainingUrl[slash] != '.') 7823 slash++; 7824 7825 // I am specifically passing `null` to differentiate it vs empty string 7826 // so in your ctor, `items` means new T(null) and `items/` means new T("") 7827 auto ident = remainingUrl.length == 0 ? null : remainingUrl[0 .. slash]; 7828 // so if it is the last item, the dot can be used to load an alternative view 7829 // otherwise tho the dot is considered part of the identifier 7830 // FIXME 7831 7832 // again notice "" vs null here! 7833 if(slash == remainingUrl.length) 7834 remainingUrl = null; 7835 else 7836 remainingUrl = remainingUrl[slash + 1 .. $]; 7837 7838 return ident; 7839 } 7840 7841 enum AddTrailingSlash; 7842 7843 private auto serveApiInternal(T)(string urlPrefix) { 7844 7845 import arsd.dom; 7846 import arsd.jsvar; 7847 7848 static bool internalHandler(Presenter)(string urlPrefix, Cgi cgi, Presenter presenter, immutable void* details) { 7849 string remainingUrl = cgi.pathInfo[urlPrefix.length .. $]; 7850 7851 try { 7852 // see duplicated code below by searching subresource_ctor 7853 // also see mustNotBeSetFromWebParams 7854 7855 static if(is(typeof(T.__ctor) P == __parameters)) { 7856 P params; 7857 7858 static foreach(pidx, param; P) { 7859 static if(is(param : Cgi)) { 7860 static assert(!is(param == immutable)); 7861 cast() params[pidx] = cgi; 7862 } else static if(is(param == Session!D, D)) { 7863 static assert(!is(param == immutable)); 7864 cast() params[pidx] = cgi.getSessionObject!D(); 7865 7866 } else { 7867 static if(hasIfCalledFromWeb!(__traits(getAttributes, P[pidx .. pidx + 1]))) { 7868 static foreach(uda; __traits(getAttributes, P[pidx .. pidx + 1])) { 7869 static if(is(uda == ifCalledFromWeb!func, alias func)) { 7870 static if(is(typeof(func(cgi)))) 7871 params[pidx] = func(cgi); 7872 else 7873 params[pidx] = func(); 7874 } 7875 } 7876 } else { 7877 7878 static if(__traits(compiles, { params[pidx] = param.getAutomaticallyForCgi(cgi); } )) { 7879 params[pidx] = param.getAutomaticallyForCgi(cgi); 7880 } else static if(is(param == string)) { 7881 auto ident = nextPieceFromSlash(remainingUrl); 7882 params[pidx] = ident; 7883 } else static assert(0, "illegal type for subresource " ~ param.stringof); 7884 } 7885 } 7886 } 7887 7888 auto obj = new T(params); 7889 } else { 7890 auto obj = new T(); 7891 } 7892 7893 return internalHandlerWithObject(obj, remainingUrl, cgi, presenter); 7894 } catch(Throwable t) { 7895 switch(cgi.request("format", "html")) { 7896 case "html": 7897 static void dummy() {} 7898 presenter.presentExceptionAsHtml!(dummy)(cgi, t, &dummy); 7899 return true; 7900 case "json": 7901 var envelope = var.emptyObject; 7902 envelope.success = false; 7903 envelope.result = null; 7904 envelope.error = t.toString(); 7905 cgi.setResponseContentType("application/json"); 7906 cgi.write(envelope.toJson(), true); 7907 return true; 7908 default: 7909 throw t; 7910 // return true; 7911 } 7912 // return true; 7913 } 7914 7915 assert(0); 7916 } 7917 7918 static bool internalHandlerWithObject(T, Presenter)(T obj, string remainingUrl, Cgi cgi, Presenter presenter) { 7919 7920 obj.initialize(cgi); 7921 7922 /+ 7923 Overload rules: 7924 Any unique combination of HTTP verb and url path can be dispatched to function overloads 7925 statically. 7926 7927 Moreover, some args vs no args can be overloaded dynamically. 7928 +/ 7929 7930 auto methodNameFromUrl = nextPieceFromSlash(remainingUrl); 7931 /+ 7932 auto orig = remainingUrl; 7933 assert(0, 7934 (orig is null ? "__null" : orig) 7935 ~ " .. " ~ 7936 (methodNameFromUrl is null ? "__null" : methodNameFromUrl)); 7937 +/ 7938 7939 if(methodNameFromUrl is null) 7940 methodNameFromUrl = "__null"; 7941 7942 string hack = to!string(cgi.requestMethod) ~ " " ~ methodNameFromUrl; 7943 7944 if(remainingUrl.length) 7945 hack ~= "/"; 7946 7947 switch(hack) { 7948 static foreach(methodName; __traits(derivedMembers, T)) 7949 static if(methodName != "__ctor") 7950 static foreach(idx, overload; __traits(getOverloads, T, methodName)) {{ 7951 static if(is(typeof(overload) P == __parameters)) 7952 static if(is(typeof(overload) R == return)) 7953 static if(__traits(getProtection, overload) == "public" || __traits(getProtection, overload) == "export") 7954 { 7955 static foreach(urlNameForMethod; urlNamesForMethod!(overload)(urlify(methodName))) 7956 case urlNameForMethod: 7957 7958 static if(is(R : WebObject)) { 7959 // if it returns a WebObject, it is considered a subresource. That means the url is dispatched like the ctor above. 7960 7961 // the only argument it is allowed to take, outside of cgi, session, and set up thingies, is a single string 7962 7963 // subresource_ctor 7964 // also see mustNotBeSetFromWebParams 7965 7966 P params; 7967 7968 static foreach(pidx, param; P) { 7969 static if(is(param : Cgi)) { 7970 static assert(!is(param == immutable)); 7971 cast() params[pidx] = cgi; 7972 } else static if(is(param == Session!D, D)) { 7973 static assert(!is(param == immutable)); 7974 cast() params[pidx] = cgi.getSessionObject!D(); 7975 } else { 7976 static if(hasIfCalledFromWeb!(__traits(getAttributes, P[pidx .. pidx + 1]))) { 7977 static foreach(uda; __traits(getAttributes, P[pidx .. pidx + 1])) { 7978 static if(is(uda == ifCalledFromWeb!func, alias func)) { 7979 static if(is(typeof(func(cgi)))) 7980 params[pidx] = func(cgi); 7981 else 7982 params[pidx] = func(); 7983 } 7984 } 7985 } else { 7986 7987 static if(__traits(compiles, { params[pidx] = param.getAutomaticallyForCgi(cgi); } )) { 7988 params[pidx] = param.getAutomaticallyForCgi(cgi); 7989 } else static if(is(param == string)) { 7990 auto ident = nextPieceFromSlash(remainingUrl); 7991 if(ident is null) { 7992 // trailing slash mandated on subresources 7993 cgi.setResponseLocation(cgi.pathInfo ~ "/"); 7994 return true; 7995 } else { 7996 params[pidx] = ident; 7997 } 7998 } else static assert(0, "illegal type for subresource " ~ param.stringof); 7999 } 8000 } 8001 } 8002 8003 auto nobj = (__traits(getOverloads, obj, methodName)[idx])(ident); 8004 return internalHandlerWithObject!(typeof(nobj), Presenter)(nobj, remainingUrl, cgi, presenter); 8005 } else { 8006 // 404 it if any url left - not a subresource means we don't get to play with that! 8007 if(remainingUrl.length) 8008 return false; 8009 8010 foreach(attr; __traits(getAttributes, overload)) 8011 static if(is(attr == AddTrailingSlash)) { 8012 if(remainingUrl is null) { 8013 cgi.setResponseLocation(cgi.pathInfo ~ "/"); 8014 return true; 8015 } 8016 } 8017 8018 /+ 8019 int zeroArgOverload = -1; 8020 int overloadCount = cast(int) __traits(getOverloads, T, methodName).length; 8021 bool calledWithZeroArgs = true; 8022 foreach(k, v; cgi.get) 8023 if(k != "format") { 8024 calledWithZeroArgs = false; 8025 break; 8026 } 8027 foreach(k, v; cgi.post) 8028 if(k != "format") { 8029 calledWithZeroArgs = false; 8030 break; 8031 } 8032 8033 // first, we need to go through and see if there is an empty one, since that 8034 // changes inside. But otherwise, all the stuff I care about can be done via 8035 // simple looping (other improper overloads might be flagged for runtime semantic check) 8036 // 8037 // an argument of type Cgi is ignored for these purposes 8038 static foreach(idx, overload; __traits(getOverloads, T, methodName)) {{ 8039 static if(is(typeof(overload) P == __parameters)) 8040 static if(P.length == 0) 8041 zeroArgOverload = cast(int) idx; 8042 else static if(P.length == 1 && is(P[0] : Cgi)) 8043 zeroArgOverload = cast(int) idx; 8044 }} 8045 // FIXME: static assert if there are multiple non-zero-arg overloads usable with a single http method. 8046 bool overloadHasBeenCalled = false; 8047 static foreach(idx, overload; __traits(getOverloads, T, methodName)) {{ 8048 bool callFunction = true; 8049 // there is a zero arg overload and this is NOT it, and we have zero args - don't call this 8050 if(overloadCount > 1 && zeroArgOverload != -1 && idx != zeroArgOverload && calledWithZeroArgs) 8051 callFunction = false; 8052 // if this is the zero-arg overload, obviously it cannot be called if we got any args. 8053 if(overloadCount > 1 && idx == zeroArgOverload && !calledWithZeroArgs) 8054 callFunction = false; 8055 8056 // FIXME: so if you just add ?foo it will give the error below even when. this might not be a great idea. 8057 8058 bool hadAnyMethodRestrictions = false; 8059 bool foundAcceptableMethod = false; 8060 foreach(attr; __traits(getAttributes, overload)) { 8061 static if(is(typeof(attr) == Cgi.RequestMethod)) { 8062 hadAnyMethodRestrictions = true; 8063 if(attr == cgi.requestMethod) 8064 foundAcceptableMethod = true; 8065 } 8066 } 8067 8068 if(hadAnyMethodRestrictions && !foundAcceptableMethod) 8069 callFunction = false; 8070 8071 /+ 8072 The overloads we really want to allow are the sane ones 8073 from the web perspective. Which is likely on HTTP verbs, 8074 for the most part, but might also be potentially based on 8075 some args vs zero args, or on argument names. Can't really 8076 do argument types very reliable through the web though; those 8077 should probably be different URLs. 8078 8079 Even names I feel is better done inside the function, so I'm not 8080 going to support that here. But the HTTP verbs and zero vs some 8081 args makes sense - it lets you define custom forms pretty easily. 8082 8083 Moreover, I'm of the opinion that empty overload really only makes 8084 sense on GET for this case. On a POST, it is just a missing argument 8085 exception and that should be handled by the presenter. But meh, I'll 8086 let the user define that, D only allows one empty arg thing anyway 8087 so the method UDAs are irrelevant. 8088 +/ 8089 if(callFunction) 8090 +/ 8091 switch(cgi.request("format", defaultFormat!overload())) { 8092 case "html": 8093 // a void return (or typeof(null) lol) means you, the user, is doing it yourself. Gives full control. 8094 try { 8095 auto ret = callFromCgi!(__traits(getOverloads, obj, methodName)[idx])(&(__traits(getOverloads, obj, methodName)[idx]), cgi); 8096 presenter.presentSuccessfulReturnAsHtml(cgi, ret); 8097 } catch(Throwable t) { 8098 presenter.presentExceptionAsHtml!(__traits(getOverloads, obj, methodName)[idx])(cgi, t, &(__traits(getOverloads, obj, methodName)[idx])); 8099 } 8100 return true; 8101 case "json": 8102 auto ret = callFromCgi!(__traits(getOverloads, obj, methodName)[idx])(&(__traits(getOverloads, obj, methodName)[idx]), cgi); 8103 static if(is(typeof(ret) == MultipleResponses!Types, Types...)) { 8104 var json; 8105 static foreach(index, type; Types) { 8106 if(ret.contains == index) 8107 json = ret.payload[index]; 8108 } 8109 } else { 8110 var json = ret; 8111 } 8112 var envelope = json; // var.emptyObject; 8113 /* 8114 envelope.success = true; 8115 envelope.result = json; 8116 envelope.error = null; 8117 */ 8118 cgi.setResponseContentType("application/json"); 8119 cgi.write(envelope.toJson(), true); 8120 return true; 8121 default: 8122 cgi.setResponseStatus("406 Not Acceptable"); // not exactly but sort of. 8123 return true; 8124 } 8125 //}} 8126 8127 //cgi.header("Accept: POST"); // FIXME list the real thing 8128 //cgi.setResponseStatus("405 Method Not Allowed"); // again, not exactly, but sort of. no overload matched our args, almost certainly due to http verb filtering. 8129 //return true; 8130 } 8131 } 8132 }} 8133 case "script.js": 8134 cgi.setResponseContentType("text/javascript"); 8135 cgi.gzipResponse = true; 8136 cgi.write(presenter.script(), true); 8137 return true; 8138 case "style.css": 8139 cgi.setResponseContentType("text/css"); 8140 cgi.gzipResponse = true; 8141 cgi.write(presenter.style(), true); 8142 return true; 8143 default: 8144 return false; 8145 } 8146 8147 assert(0); 8148 } 8149 return DispatcherDefinition!internalHandler(urlPrefix, false); 8150 } 8151 8152 string defaultFormat(alias method)() { 8153 bool nonConstConditionForWorkingAroundASpuriousDmdWarning = true; 8154 static foreach(attr; __traits(getAttributes, method)) { 8155 static if(is(typeof(attr) == DefaultFormat)) { 8156 if(nonConstConditionForWorkingAroundASpuriousDmdWarning) 8157 return attr.value; 8158 } 8159 } 8160 return "html"; 8161 } 8162 8163 string[] urlNamesForMethod(alias method)(string def) { 8164 auto verb = Cgi.RequestMethod.GET; 8165 bool foundVerb = false; 8166 bool foundNoun = false; 8167 static foreach(attr; __traits(getAttributes, method)) { 8168 static if(is(typeof(attr) == Cgi.RequestMethod)) { 8169 verb = attr; 8170 if(foundVerb) 8171 assert(0, "Multiple http verbs on one function is not currently supported"); 8172 foundVerb = true; 8173 } 8174 static if(is(typeof(attr) == UrlName)) { 8175 if(foundNoun) 8176 assert(0, "Multiple url names on one function is not currently supported"); 8177 foundNoun = true; 8178 def = attr.name; 8179 } 8180 } 8181 8182 if(def is null) 8183 def = "__null"; 8184 8185 string[] ret; 8186 8187 static if(is(typeof(method) R == return)) { 8188 static if(is(R : WebObject)) { 8189 def ~= "/"; 8190 foreach(v; __traits(allMembers, Cgi.RequestMethod)) 8191 ret ~= v ~ " " ~ def; 8192 } else { 8193 ret ~= to!string(verb) ~ " " ~ def; 8194 } 8195 } else static assert(0); 8196 8197 return ret; 8198 } 8199 8200 8201 enum AccessCheck { 8202 allowed, 8203 denied, 8204 nonExistant, 8205 } 8206 8207 enum Operation { 8208 show, 8209 create, 8210 replace, 8211 remove, 8212 update 8213 } 8214 8215 enum UpdateResult { 8216 accessDenied, 8217 noSuchResource, 8218 success, 8219 failure, 8220 unnecessary 8221 } 8222 8223 enum ValidationResult { 8224 valid, 8225 invalid 8226 } 8227 8228 8229 /++ 8230 The base of all REST objects, to be used with [serveRestObject] and [serveRestCollectionOf]. 8231 +/ 8232 class RestObject(CRTP) : WebObject { 8233 8234 import arsd.dom; 8235 import arsd.jsvar; 8236 8237 /// Prepare the object to be shown. 8238 void show() {} 8239 /// ditto 8240 void show(string urlId) { 8241 load(urlId); 8242 show(); 8243 } 8244 8245 ValidationResult delegate(typeof(this)) validateFromReflection; 8246 Element delegate(typeof(this)) toHtmlFromReflection; 8247 var delegate(typeof(this)) toJsonFromReflection; 8248 8249 /// Override this to provide access control to this object. 8250 AccessCheck accessCheck(string urlId, Operation operation) { 8251 return AccessCheck.allowed; 8252 } 8253 8254 ValidationResult validate() { 8255 if(validateFromReflection !is null) 8256 return validateFromReflection(this); 8257 return ValidationResult.valid; 8258 } 8259 8260 // The functions with more arguments are the low-level ones, 8261 // they forward to the ones with fewer arguments by default. 8262 8263 // POST on a parent collection - this is called from a collection class after the members are updated 8264 /++ 8265 Given a populated object, this creates a new entry. Returns the url identifier 8266 of the new object. 8267 +/ 8268 string create(scope void delegate() applyChanges) { 8269 return null; 8270 } 8271 8272 void replace() { 8273 save(); 8274 } 8275 void replace(string urlId, scope void delegate() applyChanges) { 8276 load(urlId); 8277 applyChanges(); 8278 replace(); 8279 } 8280 8281 void update(string[] fieldList) { 8282 save(); 8283 } 8284 void update(string urlId, scope void delegate() applyChanges, string[] fieldList) { 8285 load(urlId); 8286 applyChanges(); 8287 update(fieldList); 8288 } 8289 8290 void remove() {} 8291 8292 void remove(string urlId) { 8293 load(urlId); 8294 remove(); 8295 } 8296 8297 abstract void load(string urlId); 8298 abstract void save(); 8299 8300 Element toHtml() { 8301 if(toHtmlFromReflection) 8302 return toHtmlFromReflection(this); 8303 else 8304 assert(0); 8305 } 8306 8307 var toJson() { 8308 if(toJsonFromReflection) 8309 return toJsonFromReflection(this); 8310 else 8311 assert(0); 8312 } 8313 8314 /+ 8315 auto structOf(this This) { 8316 8317 } 8318 +/ 8319 } 8320 8321 // FIXME XSRF token, prolly can just put in a cookie and then it needs to be copied to header or form hidden value 8322 // https://use-the-index-luke.com/sql/partial-results/fetch-next-page 8323 8324 /++ 8325 Base class for REST collections. 8326 +/ 8327 class CollectionOf(Obj) : RestObject!(CollectionOf) { 8328 /// You might subclass this and use the cgi object's query params 8329 /// to implement a search filter, for example. 8330 /// 8331 /// FIXME: design a way to auto-generate that form 8332 /// (other than using the WebObject thing above lol 8333 // it'll prolly just be some searchParams UDA or maybe an enum. 8334 // 8335 // pagination too perhaps. 8336 // 8337 // and sorting too 8338 IndexResult index() { return IndexResult.init; } 8339 8340 string[] sortableFields() { return null; } 8341 string[] searchableFields() { return null; } 8342 8343 struct IndexResult { 8344 Obj[] results; 8345 8346 string[] sortableFields; 8347 8348 string previousPageIdentifier; 8349 string nextPageIdentifier; 8350 string firstPageIdentifier; 8351 string lastPageIdentifier; 8352 8353 int numberOfPages; 8354 } 8355 8356 override string create(scope void delegate() applyChanges) { assert(0); } 8357 override void load(string urlId) { assert(0); } 8358 override void save() { assert(0); } 8359 override void show() { 8360 index(); 8361 } 8362 override void show(string urlId) { 8363 show(); 8364 } 8365 8366 /// Proxy POST requests (create calls) to the child collection 8367 alias PostProxy = Obj; 8368 } 8369 8370 /++ 8371 Serves a REST object, similar to a Ruby on Rails resource. 8372 8373 You put data members in your class. cgi.d will automatically make something out of those. 8374 8375 It will call your constructor with the ID from the URL. This may be null. 8376 It will then populate the data members from the request. 8377 It will then call a method, if present, telling what happened. You don't need to write these! 8378 It finally returns a reply. 8379 8380 Your methods are passed a list of fields it actually set. 8381 8382 The URL mapping - despite my general skepticism of the wisdom - matches up with what most REST 8383 APIs I have used seem to follow. (I REALLY want to put trailing slashes on it though. Works better 8384 with relative linking. But meh.) 8385 8386 GET /items -> index. all values not set. 8387 GET /items/id -> get. only ID will be set, other params ignored. 8388 POST /items -> create. values set as given 8389 PUT /items/id -> replace. values set as given 8390 or POST /items/id with cgi.post["_method"] (thus urlencoded or multipart content-type) set to "PUT" to work around browser/html limitation 8391 a GET with cgi.get["_method"] (in the url) set to "PUT" will render a form. 8392 PATCH /items/id -> update. values set as given, list of changed fields passed 8393 or POST /items/id with cgi.post["_method"] == "PATCH" 8394 DELETE /items/id -> destroy. only ID guaranteed to be set 8395 or POST /items/id with cgi.post["_method"] == "DELETE" 8396 8397 Following the stupid convention, there will never be a trailing slash here, and if it is there, it will 8398 redirect you away from it. 8399 8400 API clients should set the `Accept` HTTP header to application/json or the cgi.get["_format"] = "json" var. 8401 8402 I will also let you change the default, if you must. 8403 8404 // One add-on is validation. You can issue a HTTP GET to a resource with _method = VALIDATE to check potential changes. 8405 8406 You can define sub-resources on your object inside the object. These sub-resources are also REST objects 8407 that follow the same thing. They may be individual resources or collections themselves. 8408 8409 Your class is expected to have at least the following methods: 8410 8411 FIXME: i kinda wanna add a routes object to the initialize call 8412 8413 create 8414 Create returns the new address on success, some code on failure. 8415 show 8416 index 8417 update 8418 remove 8419 8420 You will want to be able to customize the HTTP, HTML, and JSON returns but generally shouldn't have to - the defaults 8421 should usually work. The returned JSON will include a field "href" on all returned objects along with "id". Or omething like that. 8422 8423 Usage of this function will add a dependency on [arsd.dom] and [arsd.jsvar]. 8424 8425 NOT IMPLEMENTED 8426 8427 8428 Really, a collection is a resource with a bunch of subresources. 8429 8430 GET /items 8431 index because it is GET on the top resource 8432 8433 GET /items/foo 8434 item but different than items? 8435 8436 class Items { 8437 8438 } 8439 8440 ... but meh, a collection can be automated. not worth making it 8441 a separate thing, let's look at a real example. Users has many 8442 items and a virtual one, /users/current. 8443 8444 the individual users have properties and two sub-resources: 8445 session, which is just one, and comments, a collection. 8446 8447 class User : RestObject!() { // no parent 8448 int id; 8449 string name; 8450 8451 // the default implementations of the urlId ones is to call load(that_id) then call the arg-less one. 8452 // but you can override them to do it differently. 8453 8454 // any member which is of type RestObject can be linked automatically via href btw. 8455 8456 void show() {} 8457 void show(string urlId) {} // automated! GET of this specific thing 8458 void create() {} // POST on a parent collection - this is called from a collection class after the members are updated 8459 void replace(string urlId) {} // this is the PUT; really, it just updates all fields. 8460 void update(string urlId, string[] fieldList) {} // PATCH, it updates some fields. 8461 void remove(string urlId) {} // DELETE 8462 8463 void load(string urlId) {} // the default implementation of show() populates the id, then 8464 8465 this() {} 8466 8467 mixin Subresource!Session; 8468 mixin Subresource!Comment; 8469 } 8470 8471 class Session : RestObject!() { 8472 // the parent object may not be fully constructed/loaded 8473 this(User parent) {} 8474 8475 } 8476 8477 class Comment : CollectionOf!Comment { 8478 this(User parent) {} 8479 } 8480 8481 class Users : CollectionOf!User { 8482 // but you don't strictly need ANYTHING on a collection; it will just... collect. Implement the subobjects. 8483 void index() {} // GET on this specific thing; just like show really, just different name for the different semantics. 8484 User create() {} // You MAY implement this, but the default is to create a new object, populate it from args, and then call create() on the child 8485 } 8486 8487 +/ 8488 auto serveRestObject(T)(string urlPrefix) { 8489 assert(urlPrefix[0] == '/'); 8490 assert(urlPrefix[$ - 1] != '/', "Do NOT use a trailing slash on REST objects."); 8491 static bool internalHandler(Presenter)(string urlPrefix, Cgi cgi, Presenter presenter, immutable void* details) { 8492 string url = cgi.pathInfo[urlPrefix.length .. $]; 8493 8494 if(url.length && url[$ - 1] == '/') { 8495 // remove the final slash... 8496 cgi.setResponseLocation(cgi.scriptName ~ cgi.pathInfo[0 .. $ - 1]); 8497 return true; 8498 } 8499 8500 return restObjectServeHandler!T(cgi, presenter, url); 8501 } 8502 return DispatcherDefinition!internalHandler(urlPrefix, false); 8503 } 8504 8505 /+ 8506 /// Convenience method for serving a collection. It will be named the same 8507 /// as type T, just with an s at the end. If you need any further, just 8508 /// write the class yourself. 8509 auto serveRestCollectionOf(T)(string urlPrefix) { 8510 assert(urlPrefix[0] == '/'); 8511 mixin(`static class `~T.stringof~`s : CollectionOf!(T) {}`); 8512 return serveRestObject!(mixin(T.stringof ~ "s"))(urlPrefix); 8513 } 8514 +/ 8515 8516 bool restObjectServeHandler(T, Presenter)(Cgi cgi, Presenter presenter, string url) { 8517 string urlId = null; 8518 if(url.length && url[0] == '/') { 8519 // asking for a subobject 8520 urlId = url[1 .. $]; 8521 foreach(idx, ch; urlId) { 8522 if(ch == '/') { 8523 urlId = urlId[0 .. idx]; 8524 break; 8525 } 8526 } 8527 } 8528 8529 // FIXME handle other subresources 8530 8531 static if(is(T : CollectionOf!(C), C)) { 8532 if(urlId !is null) { 8533 return restObjectServeHandler!(C, Presenter)(cgi, presenter, url); // FIXME? urlId); 8534 } 8535 } 8536 8537 // FIXME: support precondition failed, if-modified-since, expectation failed, etc. 8538 8539 auto obj = new T(); 8540 obj.toHtmlFromReflection = delegate(t) { 8541 import arsd.dom; 8542 auto div = Element.make("div"); 8543 div.addClass("Dclass_" ~ T.stringof); 8544 div.dataset.url = urlId; 8545 bool first = true; 8546 static foreach(idx, memberName; __traits(derivedMembers, T)) 8547 static if(__traits(compiles, __traits(getMember, obj, memberName).offsetof)) { 8548 if(!first) div.addChild("br"); else first = false; 8549 div.appendChild(presenter.formatReturnValueAsHtml(__traits(getMember, obj, memberName))); 8550 } 8551 return div; 8552 }; 8553 obj.toJsonFromReflection = delegate(t) { 8554 import arsd.jsvar; 8555 var v = var.emptyObject(); 8556 static foreach(idx, memberName; __traits(derivedMembers, T)) 8557 static if(__traits(compiles, __traits(getMember, obj, memberName).offsetof)) { 8558 v[memberName] = __traits(getMember, obj, memberName); 8559 } 8560 return v; 8561 }; 8562 obj.validateFromReflection = delegate(t) { 8563 // FIXME 8564 return ValidationResult.valid; 8565 }; 8566 obj.initialize(cgi); 8567 // FIXME: populate reflection info delegates 8568 8569 8570 // FIXME: I am not happy with this. 8571 switch(urlId) { 8572 case "script.js": 8573 cgi.setResponseContentType("text/javascript"); 8574 cgi.gzipResponse = true; 8575 cgi.write(presenter.script(), true); 8576 return true; 8577 case "style.css": 8578 cgi.setResponseContentType("text/css"); 8579 cgi.gzipResponse = true; 8580 cgi.write(presenter.style(), true); 8581 return true; 8582 default: 8583 // intentionally blank 8584 } 8585 8586 8587 8588 8589 static void applyChangesTemplate(Obj)(Cgi cgi, Obj obj) { 8590 static foreach(idx, memberName; __traits(derivedMembers, Obj)) 8591 static if(__traits(compiles, __traits(getMember, obj, memberName).offsetof)) { 8592 __traits(getMember, obj, memberName) = cgi.request(memberName, __traits(getMember, obj, memberName)); 8593 } 8594 } 8595 void applyChanges() { 8596 applyChangesTemplate(cgi, obj); 8597 } 8598 8599 string[] modifiedList; 8600 8601 void writeObject(bool addFormLinks) { 8602 if(cgi.request("format") == "json") { 8603 cgi.setResponseContentType("application/json"); 8604 cgi.write(obj.toJson().toString, true); 8605 } else { 8606 auto container = presenter.htmlContainer(); 8607 if(addFormLinks) { 8608 static if(is(T : CollectionOf!(C), C)) 8609 container.appendHtml(` 8610 <form> 8611 <button type="submit" name="_method" value="POST">Create New</button> 8612 </form> 8613 `); 8614 else 8615 container.appendHtml(` 8616 <form> 8617 <button type="submit" name="_method" value="PATCH">Edit</button> 8618 <button type="submit" name="_method" value="DELETE">Delete</button> 8619 </form> 8620 `); 8621 } 8622 container.appendChild(obj.toHtml()); 8623 cgi.write(container.parentDocument.toString, true); 8624 } 8625 } 8626 8627 // FIXME: I think I need a set type in here.... 8628 // it will be nice to pass sets of members. 8629 8630 try 8631 switch(cgi.requestMethod) { 8632 case Cgi.RequestMethod.GET: 8633 // I could prolly use template this parameters in the implementation above for some reflection stuff. 8634 // sure, it doesn't automatically work in subclasses... but I instantiate here anyway... 8635 8636 // automatic forms here for usable basic auto site from browser. 8637 // even if the format is json, it could actually send out the links and formats, but really there i'ma be meh. 8638 switch(cgi.request("_method", "GET")) { 8639 case "GET": 8640 static if(is(T : CollectionOf!(C), C)) { 8641 auto results = obj.index(); 8642 if(cgi.request("format", "html") == "html") { 8643 auto container = presenter.htmlContainer(); 8644 auto html = presenter.formatReturnValueAsHtml(results.results); 8645 container.appendHtml(` 8646 <form> 8647 <button type="submit" name="_method" value="POST">Create New</button> 8648 </form> 8649 `); 8650 8651 container.appendChild(html); 8652 cgi.write(container.parentDocument.toString, true); 8653 } else { 8654 cgi.setResponseContentType("application/json"); 8655 import arsd.jsvar; 8656 var json = var.emptyArray; 8657 foreach(r; results.results) { 8658 var o = var.emptyObject; 8659 static foreach(idx, memberName; __traits(derivedMembers, typeof(r))) 8660 static if(__traits(compiles, __traits(getMember, r, memberName).offsetof)) { 8661 o[memberName] = __traits(getMember, r, memberName); 8662 } 8663 8664 json ~= o; 8665 } 8666 cgi.write(json.toJson(), true); 8667 } 8668 } else { 8669 obj.show(urlId); 8670 writeObject(true); 8671 } 8672 break; 8673 case "PATCH": 8674 obj.load(urlId); 8675 goto case; 8676 case "PUT": 8677 case "POST": 8678 // an editing form for the object 8679 auto container = presenter.htmlContainer(); 8680 static if(__traits(compiles, () { auto o = new obj.PostProxy(); })) { 8681 auto form = (cgi.request("_method") == "POST") ? presenter.createAutomaticFormForObject(new obj.PostProxy()) : presenter.createAutomaticFormForObject(obj); 8682 } else { 8683 auto form = presenter.createAutomaticFormForObject(obj); 8684 } 8685 form.attrs.method = "POST"; 8686 form.setValue("_method", cgi.request("_method", "GET")); 8687 container.appendChild(form); 8688 cgi.write(container.parentDocument.toString(), true); 8689 break; 8690 case "DELETE": 8691 // FIXME: a delete form for the object (can be phrased "are you sure?") 8692 auto container = presenter.htmlContainer(); 8693 container.appendHtml(` 8694 <form method="POST"> 8695 Are you sure you want to delete this item? 8696 <input type="hidden" name="_method" value="DELETE" /> 8697 <input type="submit" value="Yes, Delete It" /> 8698 </form> 8699 8700 `); 8701 cgi.write(container.parentDocument.toString(), true); 8702 break; 8703 default: 8704 cgi.write("bad method\n", true); 8705 } 8706 break; 8707 case Cgi.RequestMethod.POST: 8708 // this is to allow compatibility with HTML forms 8709 switch(cgi.request("_method", "POST")) { 8710 case "PUT": 8711 goto PUT; 8712 case "PATCH": 8713 goto PATCH; 8714 case "DELETE": 8715 goto DELETE; 8716 case "POST": 8717 static if(__traits(compiles, () { auto o = new obj.PostProxy(); })) { 8718 auto p = new obj.PostProxy(); 8719 void specialApplyChanges() { 8720 applyChangesTemplate(cgi, p); 8721 } 8722 string n = p.create(&specialApplyChanges); 8723 } else { 8724 string n = obj.create(&applyChanges); 8725 } 8726 8727 auto newUrl = cgi.scriptName ~ cgi.pathInfo ~ "/" ~ n; 8728 cgi.setResponseLocation(newUrl); 8729 cgi.setResponseStatus("201 Created"); 8730 cgi.write(`The object has been created.`); 8731 break; 8732 default: 8733 cgi.write("bad method\n", true); 8734 } 8735 // FIXME this should be valid on the collection, but not the child.... 8736 // 303 See Other 8737 break; 8738 case Cgi.RequestMethod.PUT: 8739 PUT: 8740 obj.replace(urlId, &applyChanges); 8741 writeObject(false); 8742 break; 8743 case Cgi.RequestMethod.PATCH: 8744 PATCH: 8745 obj.update(urlId, &applyChanges, modifiedList); 8746 writeObject(false); 8747 break; 8748 case Cgi.RequestMethod.DELETE: 8749 DELETE: 8750 obj.remove(urlId); 8751 cgi.setResponseStatus("204 No Content"); 8752 break; 8753 default: 8754 // FIXME: OPTIONS, HEAD 8755 } 8756 catch(Throwable t) { 8757 presenter.presentExceptionAsHtml!(DUMMY)(cgi, t, null); 8758 } 8759 8760 return true; 8761 } 8762 8763 struct DUMMY {} 8764 8765 /+ 8766 struct SetOfFields(T) { 8767 private void[0][string] storage; 8768 void set(string what) { 8769 //storage[what] = 8770 } 8771 void unset(string what) {} 8772 void setAll() {} 8773 void unsetAll() {} 8774 bool isPresent(string what) { return false; } 8775 } 8776 +/ 8777 8778 /+ 8779 enum readonly; 8780 enum hideonindex; 8781 +/ 8782 8783 /++ 8784 Serves a static file. To be used with [dispatcher]. 8785 8786 See_Also: [serveApi], [serveRestObject], [dispatcher], [serveRedirect] 8787 +/ 8788 auto serveStaticFile(string urlPrefix, string filename = null, string contentType = null) { 8789 // https://baus.net/on-tcp_cork/ 8790 // man 2 sendfile 8791 assert(urlPrefix[0] == '/'); 8792 if(filename is null) 8793 filename = urlPrefix[1 .. $]; 8794 if(contentType is null) { 8795 contentType = contentTypeFromFileExtension(filename); 8796 } 8797 8798 static struct DispatcherDetails { 8799 string filename; 8800 string contentType; 8801 } 8802 8803 static bool internalHandler(string urlPrefix, Cgi cgi, Object presenter, DispatcherDetails details) { 8804 if(details.contentType.indexOf("image/") == 0) 8805 cgi.setCache(true); 8806 cgi.setResponseContentType(details.contentType); 8807 cgi.write(std.file.read(details.filename), true); 8808 return true; 8809 } 8810 return DispatcherDefinition!(internalHandler, DispatcherDetails)(urlPrefix, true, DispatcherDetails(filename, contentType)); 8811 } 8812 8813 string contentTypeFromFileExtension(string filename) { 8814 if(filename.endsWith(".png")) 8815 return "image/png"; 8816 if(filename.endsWith(".svg")) 8817 return "image/svg+xml"; 8818 if(filename.endsWith(".jpg")) 8819 return "image/jpeg"; 8820 if(filename.endsWith(".html")) 8821 return "text/html"; 8822 if(filename.endsWith(".css")) 8823 return "text/css"; 8824 if(filename.endsWith(".js")) 8825 return "application/javascript"; 8826 if(filename.endsWith(".mp3")) 8827 return "audio/mpeg"; 8828 return null; 8829 } 8830 8831 /// This serves a directory full of static files, figuring out the content-types from file extensions. 8832 /// It does not let you to descend into subdirectories (or ascend out of it, of course) 8833 auto serveStaticFileDirectory(string urlPrefix, string directory = null) { 8834 assert(urlPrefix[0] == '/'); 8835 assert(urlPrefix[$-1] == '/'); 8836 8837 static struct DispatcherDetails { 8838 string directory; 8839 } 8840 8841 if(directory is null) 8842 directory = urlPrefix[1 .. $]; 8843 8844 assert(directory[$-1] == '/'); 8845 8846 static bool internalHandler(string urlPrefix, Cgi cgi, Object presenter, DispatcherDetails details) { 8847 auto file = cgi.pathInfo[urlPrefix.length .. $]; 8848 if(file.indexOf("/") != -1 || file.indexOf("\\") != -1) 8849 return false; 8850 8851 auto contentType = contentTypeFromFileExtension(file); 8852 8853 auto fn = details.directory ~ file; 8854 if(std.file.exists(fn)) { 8855 //if(contentType.indexOf("image/") == 0) 8856 //cgi.setCache(true); 8857 //else if(contentType.indexOf("audio/") == 0) 8858 cgi.setCache(true); 8859 cgi.setResponseContentType(contentType); 8860 cgi.write(std.file.read(fn), true); 8861 return true; 8862 } else { 8863 return false; 8864 } 8865 } 8866 8867 return DispatcherDefinition!(internalHandler, DispatcherDetails)(urlPrefix, false, DispatcherDetails(directory)); 8868 } 8869 8870 private static string getHttpCodeText(int code) pure nothrow @nogc { 8871 switch(code) { 8872 case 200: return "200 OK"; 8873 case 201: return "201 Created"; 8874 case 202: return "202 Accepted"; 8875 case 203: return "203 Non-Authoritative Information"; 8876 case 204: return "204 No Content"; 8877 case 205: return "205 Reset Content"; 8878 // 8879 case 300: return "300 300 Multiple Choices"; 8880 case 301: return "301 Moved Permanently"; 8881 case 302: return "302 Found"; 8882 case 303: return "303 See Other"; 8883 case 307: return "307 Temporary Redirect"; 8884 case 308: return "308 Permanent Redirect"; 8885 // 8886 // FIXME: add more common 400 ones cgi.d might return too 8887 case 400: return "400 Bad Request"; 8888 case 403: return "403 Forbidden"; 8889 case 404: return "404 Not Found"; 8890 case 405: return "405 Method Not Allowed"; 8891 case 406: return "406 Not Acceptable"; 8892 case 409: return "409 Conflict"; 8893 case 410: return "410 Gone"; 8894 // 8895 case 500: return "500 Internal Server Error"; 8896 case 501: return "501 Not Implemented"; 8897 case 502: return "502 Bad Gateway"; 8898 case 503: return "503 Service Unavailable"; 8899 // 8900 default: assert(0, "Unsupported http code"); 8901 } 8902 } 8903 8904 /++ 8905 Redirects one url to another 8906 8907 See_Also: [dispatcher], [serveStaticFile] 8908 +/ 8909 auto serveRedirect(string urlPrefix, string redirectTo, int code = 303) { 8910 assert(urlPrefix[0] == '/'); 8911 static struct DispatcherDetails { 8912 string redirectTo; 8913 string code; 8914 } 8915 8916 static bool internalHandler(string urlPrefix, Cgi cgi, Object presenter, DispatcherDetails details) { 8917 cgi.setResponseLocation(details.redirectTo, true, details.code); 8918 return true; 8919 } 8920 8921 8922 return DispatcherDefinition!(internalHandler, DispatcherDetails)(urlPrefix, true, DispatcherDetails(redirectTo, getHttpCodeText(code))); 8923 } 8924 8925 /// Used exclusively with `dispatchTo` 8926 struct DispatcherData(Presenter) { 8927 Cgi cgi; /// You can use this cgi object. 8928 Presenter presenter; /// This is the presenter from top level, and will be forwarded to the sub-dispatcher. 8929 size_t pathInfoStart; /// This is forwarded to the sub-dispatcher. It may be marked private later, or at least read-only. 8930 } 8931 8932 /++ 8933 Dispatches the URL to a specific function. 8934 +/ 8935 auto handleWith(alias handler)(string urlPrefix) { 8936 // cuz I'm too lazy to do it better right now 8937 static class Hack : WebObject { 8938 static import std.traits; 8939 @UrlName("") 8940 auto handle(std.traits.Parameters!handler args) { 8941 return handler(args); 8942 } 8943 } 8944 8945 return urlPrefix.serveApiInternal!Hack; 8946 } 8947 8948 /++ 8949 Dispatches the URL (and anything under it) to another dispatcher function. The function should look something like this: 8950 8951 --- 8952 bool other(DD)(DD dd) { 8953 return dd.dispatcher!( 8954 "/whatever".serveRedirect("/success"), 8955 "/api/".serveApi!MyClass 8956 ); 8957 } 8958 --- 8959 8960 The `DD` in there will be an instance of [DispatcherData] which you can inspect, or forward to another dispatcher 8961 here. It is a template to account for any Presenter type, so you can do compile-time analysis in your presenters. 8962 Or, of course, you could just use the exact type in your own code. 8963 8964 You return true if you handle the given url, or false if not. Just returning the result of [dispatcher] will do a 8965 good job. 8966 8967 8968 +/ 8969 auto dispatchTo(alias handler)(string urlPrefix) { 8970 assert(urlPrefix[0] == '/'); 8971 assert(urlPrefix[$-1] != '/'); 8972 static bool internalHandler(Presenter)(string urlPrefix, Cgi cgi, Presenter presenter, const void* details) { 8973 return handler(DispatcherData!Presenter(cgi, presenter, urlPrefix.length)); 8974 } 8975 8976 return DispatcherDefinition!(internalHandler)(urlPrefix, false); 8977 } 8978 8979 /+ 8980 /++ 8981 See [serveStaticFile] if you want to serve a file off disk. 8982 +/ 8983 auto serveStaticData(string urlPrefix, const(void)[] data, string contentType) { 8984 8985 } 8986 +/ 8987 8988 /++ 8989 A URL dispatcher. 8990 8991 --- 8992 if(cgi.dispatcher!( 8993 "/api/".serveApi!MyApiClass, 8994 "/objects/lol".serveRestObject!MyRestObject, 8995 "/file.js".serveStaticFile, 8996 "/admin/".dispatchTo!adminHandler 8997 )) return; 8998 --- 8999 9000 9001 You define a series of url prefixes followed by handlers. 9002 9003 [dispatchTo] will send the request to another function for handling. 9004 You may want to do different pre- and post- processing there, for example, 9005 an authorization check and different page layout. You can use different 9006 presenters and different function chains. NOT IMPLEMENTED 9007 +/ 9008 template dispatcher(definitions...) { 9009 bool dispatcher(Presenter)(Cgi cgi, Presenter presenterArg = null) { 9010 static if(is(Presenter == typeof(null))) { 9011 static class GenericWebPresenter : WebPresenter!(GenericWebPresenter) {} 9012 auto presenter = new GenericWebPresenter(); 9013 } else 9014 alias presenter = presenterArg; 9015 9016 return dispatcher(DispatcherData!(typeof(presenter))(cgi, presenter, 0)); 9017 } 9018 9019 bool dispatcher(DispatcherData)(DispatcherData dispatcherData) if(!is(DispatcherData : Cgi)) { 9020 // I can prolly make this more efficient later but meh. 9021 static foreach(definition; definitions) { 9022 if(definition.rejectFurther) { 9023 if(dispatcherData.cgi.pathInfo[dispatcherData.pathInfoStart .. $] == definition.urlPrefix) { 9024 auto ret = definition.handler( 9025 dispatcherData.cgi.pathInfo[0 .. dispatcherData.pathInfoStart + definition.urlPrefix.length], 9026 dispatcherData.cgi, dispatcherData.presenter, definition.details); 9027 if(ret) 9028 return true; 9029 } 9030 } else if( 9031 dispatcherData.cgi.pathInfo[dispatcherData.pathInfoStart .. $].startsWith(definition.urlPrefix) && 9032 // cgi.d dispatcher urls must be complete or have a /; 9033 // "foo" -> thing should NOT match "foobar", just "foo" or "foo/thing" 9034 (definition.urlPrefix[$-1] == '/' || (dispatcherData.pathInfoStart + definition.urlPrefix.length) == dispatcherData.cgi.pathInfo.length 9035 || dispatcherData.cgi.pathInfo[dispatcherData.pathInfoStart + definition.urlPrefix.length] == '/') 9036 ) { 9037 auto ret = definition.handler( 9038 dispatcherData.cgi.pathInfo[0 .. dispatcherData.pathInfoStart + definition.urlPrefix.length], 9039 dispatcherData.cgi, dispatcherData.presenter, definition.details); 9040 if(ret) 9041 return true; 9042 } 9043 } 9044 return false; 9045 } 9046 } 9047 9048 }); 9049 9050 /+ 9051 /++ 9052 This is the beginnings of my web.d 2.0 - it dispatches web requests to a class object. 9053 9054 It relies on jsvar.d and dom.d. 9055 9056 9057 You can get javascript out of it to call. The generated functions need to look 9058 like 9059 9060 function name(a,b,c,d,e) { 9061 return _call("name", {"realName":a,"sds":b}); 9062 } 9063 9064 And _call returns an object you can call or set up or whatever. 9065 +/ 9066 bool apiDispatcher()(Cgi cgi) { 9067 import arsd.jsvar; 9068 import arsd.dom; 9069 } 9070 +/ 9071 /* 9072 Copyright: Adam D. Ruppe, 2008 - 2019 9073 License: <a href="http://www.boost.org/LICENSE_1_0.txt">Boost License 1.0</a>. 9074 Authors: Adam D. Ruppe 9075 9076 Copyright Adam D. Ruppe 2008 - 2019. 9077 Distributed under the Boost Software License, Version 1.0. 9078 (See accompanying file LICENSE_1_0.txt or copy at 9079 http://www.boost.org/LICENSE_1_0.txt) 9080 */